Encora Logo

Encora

Data Engineer (ETL Pipeline)

Sorry, this job was removed at 10:41 p.m. (IST) on Monday, Feb 17, 2025
Be an Early Applicant
India
India

Important Information

Location: India

Experience: 8+ years

Job Mode: Full-time

Job Summary

The Data Engineer will play a critical role in the data modernization project by focusing on the design, automation, and optimization of ETL pipelines. This role will primarily involve ensuring seamless integration with Snowflake and PostgreSQL and implementing robust data transformation workflows. The Data Engineer will collaborate closely with the Airflow Specialist and the Data Architect to maintain alignment with the overall data architecture while focusing on delivering high-performance data pipelines.

Responsibilities and Duties

  1. ETL Pipeline Development & Automation:
    • Design, develop, and automate ETL pipelines to migrate data from legacy systems to modern platforms.
    • Leverage tools such as AWS Glue, dbt, and Python to support data ingestion, transformation, and integration.
    • Maintain high-quality, scalable pipelines capable of processing large datasets efficiently.
  2. Data Transformation & Integration:
    • Implement data transformation workflows to prepare data for analytical workloads in Snowflake and PostgreSQL environments.
    • Ensure data consistency and integrity during transformations by leveraging best practices in schema enforcement and validation.
  3. Collaboration & Alignment:
    • Work closely with the Airflow Specialist to align ETL workflows with orchestration pipelines.
    • Collaborate with the Data Architect to ensure data models and pipelines align with overall system design and project goals.
  4. Process Optimization:
    • Identify opportunities for optimizing ETL processes to enhance performance and reduce costs.
    • Implement change data capture (CDC) mechanisms and incremental updates to ensure timely data availability.
  5. Data Quality & Monitoring:
    • Support the integration of automated validation tools, such as Great Expectations, into ETL workflows.
    • Monitor ETL processes to ensure consistent performance and timely issue resolution.

Qualifications and Skills

  • 8+ years of experience in data engineering, with a focus on building and automating ETL pipelines.
  • Hands-on expertise with AWS Glue, Python, and dbt for data transformation and pipeline development.
  • Proven experience integrating data workflows into Snowflake and PostgreSQL.
  • Strong understanding of modern data architecture principles and ETL best practices.
  • Familiarity with data governance tools such as DataHub and validation frameworks like Great Expectations.
  • 5+ years of experience in data engineering, with a focus on building and automating ETL pipelines.
  • Hands-on expertise with AWS Glue, Python, and dbt for data transformation and pipeline development.
  • Proven experience integrating data workflows into Snowflake and PostgreSQL.
  • Strong understanding of modern data architecture principles and ETL best practices.
  • Familiarity with data governance tools such as DataHub and validation frameworks like Great Expectations.

Additional Requirements

  • Experience with change data capture (CDC) and real-time data processing mechanisms.
  • Familiarity with Apache Spark for large-scale data transformation tasks.
  • Proficiency in integrating ETL workflows into Airflow orchestrations.
  • Strong analytical skills for identifying performance improvement opportunities in data workflows.

About Encora

Encora is a global company that offers Software and Digital Engineering solutions. Our practices include Cloud Services, Product Engineering & Application Modernization, Data & Analytics, Digital Experience & Design Services, DevSecOps, Cybersecurity, Quality Engineering, AI & LLM Engineering, among others.

At Encora, we hire professionals based solely on their skills and do not discriminate based on age, disability, religion, gender, sexual orientation, socioeconomic status, or nationality.

Similar Jobs

16 Minutes Ago
Hybrid
Navi Mumbai, Thane, Maharashtra, IND
Senior level
Senior level
Enterprise Web • Fintech • Financial Services
The role involves building and maintaining data technologies for advanced business analytics, including ETL processes, and ensuring high-quality insights for internal stakeholders.
Top Skills: .NetAWSC#Morningstar Bi PlatformPower BIPythonSalesforceSQLTableau
30 Minutes Ago
Bangalore, Bengaluru, Karnataka, IND
Mid level
Mid level
eCommerce • Fintech • Payments • Software • Financial Services
The Facilities Coordinator ensures smooth office operations, manages visitor flow, oversees local budgets, implements safety programs, and coordinates maintenance.
Top Skills: Facilities Management SoftwareTicketing Systems
31 Minutes Ago
Hybrid
Gurgaon, Gurugram, Haryana, IND
Senior level
Senior level
Artificial Intelligence • Healthtech • Professional Services • Analytics • Consulting
Lead the delivery of Master Data Management (MDM) solutions across multiple client engagements, ensuring design quality and client satisfaction. Develop data governance frameworks, implement MDM strategies, and drive continuous improvement with a global team. Act as a liaison with senior leadership and manage overall project delivery, including staffing and planning.
Top Skills: AWSAzureInformatica MdmReltioSnowflake

What you need to know about the Kolkata Tech Scene

When considering the industries shaping India's tech scene, gaming might not immediately come to mind. However, in the last decade, increased internet usage and greater access to mobile devices have catapulted the industry to new heights, with Kolkata-based companies like Virtualinfocom, Red Apple Technologies and Digitoonz, at the forefront, driving the design and animation of new gaming titles for players.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account