The job requires establishing CI/CD pipelines for data workflows, building ETL/ELT processes using Azure services, ensuring platform reliability, and implementing data governance practices.
Description and Requirements
CI/CD, DevOps, and Automated Delivery (Primary Focus)
• Establish and maintain CI/CD pipeline development, deployment, and testing for data workflows using Azure DevOps or GitHub Actions.
• Implement Git-based source control practices for data engineering assets (ADF pipelines, Databricks notebooks/jobs, SQL scripts), including branching and release management across environments.
• Automate deployments with validation and rollback strategies and set up monitoring/alerting for pipeline and platform health.
• Build and maintain YAML-based pipelines and standardized build/deploy patterns (e.g., reusable templates, triggers, approvals).
• Ensure production deployments follow required change governance and audit-friendly release controls.
Data Engineering & ETL/ELT Pipelines (ADF + Databricks)
• Architect, build, and maintain ETL/ELT pipelines using Azure Data Factory and Azure Databricks (PySpark, SQL, Delta Lake).
• Optimize ingestion, transformation, and storage patterns for performance, scalability, reliability, and cost efficiency.
• Implement orchestration standards, error handling, retries, and operational guardrails to meet SLA/SLO expectations.
• Develop robust transformations/aggregations to support downstream analytics and reporting needs.
Azure Platform Operations & Reliability
• Support administration and optimization of core Azure data services used by the platform (e.g., ADF, Databricks, ADLS, Azure SQL and/or Synapse where applicable).
• Monitor platform health, performance, and cost usage; implement automation to improve efficiency and reliability.
Data Governance, Security & Compliance
• Implement data access controls, auditing, and security best practices using Azure-native capabilities (e.g., RBAC, secrets management, and governance tooling such as Purview or equivalent where applicable).
Knowledge, Skills and Abilities
Education
Experience
Knowledge and skills (general and technical)
Preferred / Nice to Have
About MetLife
Recognized on Fortune magazine's list of the "World's Most Admired Companies" and Fortune World's 25 Best Workplaces™, MetLife, through its subsidiaries and affiliates, is one of the world's leading financial services companies; providing insurance, annuities, employee benefits and asset management to individual and institutional customers. With operations in more than 40 markets, we hold leading positions in the United States, Latin America, Asia, Europe, and the Middle East.
Our purpose is simple - to help our colleagues, customers, communities, and the world at large create a more confident future. United by purpose and guided by our core values - Win Together, Do the Right Thing, Deliver Impact Over Activity, and Think Ahead - we're inspired to transform the next century in financial services. At MetLife, it's #AllTogetherPossible . Join us!
#BI-Hybrid
CI/CD, DevOps, and Automated Delivery (Primary Focus)
• Establish and maintain CI/CD pipeline development, deployment, and testing for data workflows using Azure DevOps or GitHub Actions.
• Implement Git-based source control practices for data engineering assets (ADF pipelines, Databricks notebooks/jobs, SQL scripts), including branching and release management across environments.
• Automate deployments with validation and rollback strategies and set up monitoring/alerting for pipeline and platform health.
• Build and maintain YAML-based pipelines and standardized build/deploy patterns (e.g., reusable templates, triggers, approvals).
• Ensure production deployments follow required change governance and audit-friendly release controls.
Data Engineering & ETL/ELT Pipelines (ADF + Databricks)
• Architect, build, and maintain ETL/ELT pipelines using Azure Data Factory and Azure Databricks (PySpark, SQL, Delta Lake).
• Optimize ingestion, transformation, and storage patterns for performance, scalability, reliability, and cost efficiency.
• Implement orchestration standards, error handling, retries, and operational guardrails to meet SLA/SLO expectations.
• Develop robust transformations/aggregations to support downstream analytics and reporting needs.
Azure Platform Operations & Reliability
• Support administration and optimization of core Azure data services used by the platform (e.g., ADF, Databricks, ADLS, Azure SQL and/or Synapse where applicable).
• Monitor platform health, performance, and cost usage; implement automation to improve efficiency and reliability.
Data Governance, Security & Compliance
• Implement data access controls, auditing, and security best practices using Azure-native capabilities (e.g., RBAC, secrets management, and governance tooling such as Purview or equivalent where applicable).
Knowledge, Skills and Abilities
Education
- Preferably Graduate in Computer Applications / Computer Science.
Experience
- 2-4 years relevant experience
Knowledge and skills (general and technical)
- Deep hands-on expertise in Azure Data Factory (ADF) and Azure Databricks (PySpark/SQL/Delta Lake).
- Strong coding skills in SQL and Python/PySpark, including performance tuning and operational troubleshooting.
- Proven experience implementing CI/CD pipelines and DevOps best practices for data engineering deployments using Azure DevOps or GitHub Actions.
- At least 2+ years in Azure-based environments
Preferred / Nice to Have
- Familiarity with enterprise DevOps delivery standards for Azure DevOps pipelines (build/deploy pipeline patterns and operational steps).
- Microsoft Certified: Azure Data Engineer Associate preferred (Azure Solutions Architect Expert a plus).
About MetLife
Recognized on Fortune magazine's list of the "World's Most Admired Companies" and Fortune World's 25 Best Workplaces™, MetLife, through its subsidiaries and affiliates, is one of the world's leading financial services companies; providing insurance, annuities, employee benefits and asset management to individual and institutional customers. With operations in more than 40 markets, we hold leading positions in the United States, Latin America, Asia, Europe, and the Middle East.
Our purpose is simple - to help our colleagues, customers, communities, and the world at large create a more confident future. United by purpose and guided by our core values - Win Together, Do the Right Thing, Deliver Impact Over Activity, and Think Ahead - we're inspired to transform the next century in financial services. At MetLife, it's #AllTogetherPossible . Join us!
#BI-Hybrid
Similar Jobs at MetLife
Fintech • Information Technology • Insurance • Financial Services • Big Data Analytics
The role involves designing and developing cloud-based applications, scalable microservices, and CI/CD pipelines, while collaborating with teams and stakeholders.
Top Skills:
Advanced JavaAngularjsBambooCi/CdCore JavaDockerEs6GradleGulpJavaScriptJenkinsMavenMicro ServicesMongoDBNode.jsNosql DatabasesReact NativeReactRedisSpring BootSpring CloudSpring FrameworkWebpack
Fintech • Information Technology • Insurance • Financial Services • Big Data Analytics
As a Big Data Engineer, you will build and implement data ingestion processes, collaborate with the data science team, monitor performance, and ensure data security while utilizing a variety of big data tools and technologies.
Top Skills:
DatabricksDelta LakeFlumeHdfsHiveImpalaJavaKafkaKerberosOoziePigPythonScalaSentrySparkSQLSqoopZookeeper
Fintech • Information Technology • Insurance • Financial Services • Big Data Analytics
The role involves designing and developing cloud-based applications, scalable microservices, and CI/CD pipelines, while collaborating with teams and stakeholders.
Top Skills:
Advanced JavaAngularjsBambooCi/CdCore JavaDockerEs6GradleGulpJavaScriptJenkinsMavenMicro ServicesMongoDBNode.jsNosql DatabasesReact NativeReactRedisSpring BootSpring CloudSpring FrameworkWebpack
What you need to know about the Kolkata Tech Scene
When considering the industries shaping India's tech scene, gaming might not immediately come to mind. However, in the last decade, increased internet usage and greater access to mobile devices have catapulted the industry to new heights, with Kolkata-based companies like Virtualinfocom, Red Apple Technologies and Digitoonz, at the forefront, driving the design and animation of new gaming titles for players.
