The Data Engineer will design and maintain data pipelines, optimize workflows, ensure data quality, and implement CI/CD practices using AWS and other tools.
We are looking for an experienced Data Engineer to design, build, and optimize scalable data pipelines and analytics solutions. The ideal candidate will have strong hands-on experience with cloud-based data platforms, big data processing, and analytics engineering best practices.
RequirementsKey Responsibilities
- Design, develop, and maintain scalable data pipelines using AWS EMR, PySpark, and Python
- Work closely with analytics and business teams to build reliable and high-performance data models
- Optimize data processing workflows for performance, reliability, and cost
- Ensure data quality, integrity, and consistency across data platforms
- Implement CI/CD practices for data pipelines using modern DevOps tools
- Collaborate using version control and follow best engineering practices
- Strong experience with AWS (especially EMR)
- Hands-on experience in Python and PySpark
- Strong SQL and analytical skills
- Experience with data pipeline orchestration and tooling
- Working knowledge of GitHub for version control
- Experience with Jenkins or similar CI/CD tools
- Familiarity with dbt
- Experience with Terraform or other Infrastructure-as-Code tools
- Deeper hands-on expertise with dbt
- Exposure to cloud cost optimization and data governance
- Experience working in Agile environments
- Strong communication and stakeholder collaboration skills
BenefitsWhat We Offer
- Opportunity to work on large-scale, modern data platforms
- Collaborative and engineering-driven culture
- Competitive compensation and benefits
Top Skills
Aws Emr
Dbt
Git
Jenkins
Pyspark
Python
SQL
Terraform
Similar Jobs
Big Data • Fintech • Information Technology • Business Intelligence • Financial Services • Cybersecurity • Big Data Analytics
The Insights Data Analyst will support operational and analytical functions, improve customer experience through data analysis, and develop insights-driven reports and dashboards, collaborating across teams to address business questions and drive improvements.
Top Skills:
DomoJavaScriptPower BIPythonRSpssTableau
Cloud • Information Technology • Security • Software • Cybersecurity
The Senior Named Account Executive will drive sales in BFSI enterprise accounts, manage relationships, negotiate contracts, and achieve revenue targets.
Top Skills:
CdnCloudflareNetwork SecuritySaaS
Artificial Intelligence • Hardware • Information Technology • Security • Software • Cybersecurity • Big Data Analytics
Join the Product Primary Data team as a DevOps Developer to build, automate, and maintain CI/CD infrastructure for MDM solutions, ensuring reliable and secure platform management, optimizing workflows, and promoting DevOps best practices.
Top Skills:
APIsInformatica MdmMulesoftOicOracle
What you need to know about the Kolkata Tech Scene
When considering the industries shaping India's tech scene, gaming might not immediately come to mind. However, in the last decade, increased internet usage and greater access to mobile devices have catapulted the industry to new heights, with Kolkata-based companies like Virtualinfocom, Red Apple Technologies and Digitoonz, at the forefront, driving the design and animation of new gaming titles for players.
