Design, build and optimize ETL pipelines and cloud data workflows using Python, PySpark, AWS Glue and Lambda. Implement data models and schemas in PL/SQL/PostgreSQL, integrate AWS services (SNS, Step Functions), and troubleshoot pipelines for performance, scalability and reliability.
Location: Bangalore
Experience: 6 - 9 Years
We are looking for a Data Engineer with strong expertise in Python, PySpark, AWS Glue, and Lambda. The ideal candidate should have experience in data processing, transformation, and cloud-based data workflows. A strong foundation in PL/SQL and PostgreSQL is required, and experience with AWS services such as SNS and Step Functions, along with Java API development, would be a plus.
Key Responsibilities:- Develop and optimize ETL pipelines using AWS Glue, Lambda, and PySpark.
- Work with large-scale data processing and transformation using Python and Spark.
- Design and implement data workflows for structured and unstructured data.
- Optimize queries and data storage for performance and cost efficiency.
- Develop and maintain data models, schemas, and database structures using PL/SQL and PostgreSQL.
- Integrate AWS services such as SNS, Step Functions, and other cloud-based workflows.
- Collaborate with cross-functional teams to support business data needs.
- Troubleshoot and optimize data pipelines for scalability and reliability.
✅ Python & PySpark – Strong hands-on experience
✅ AWS Glue & Lambda – Experience in serverless data processing
✅ PL/SQL & PostgreSQL – Basic database development and querying skills
SNS & Step Functions – AWS workflow automation
Java API – Experience in API development and integration
Similar Jobs
Agency • Information Technology
Design, build, and maintain cloud data pipelines and streaming solutions using Java, Kafka, AWS services, SQL/warehouse technologies, and Python. Implement CI/CD, automated testing, and resilient microservice-based systems. Collaborate in a small agile team and ensure secure identity/authentication controls.
Top Skills:
Apache KafkaAurora PostgresqlAws LambdaAws S3Aws SnsAws SqsCi/CdDockerJavaJenkinsJunitKubernetesMicroservicesMockitoPytestPythonSnowflakeSpring BootSQL
Agency • Information Technology
Design, develop, and coordinate cloud data solutions using Snowflake, Python, and AWS. Support product releases, deployments, testing, and post-install support. Collaborate with architects and tech leads, troubleshoot issues, and maintain documentation. Contribute to agile squads, adopt new technologies, and enable Fidelity Wealth business data needs.
Top Skills:
AWSCi/CdControl-MCopilotEvent Driven ArchitectureJenkinsPl-SqlPythonRelational DatabasesServerless FrameworkSnowflake
Agency • Information Technology
Design, build, and maintain cloud data pipelines and architectures using SQL, Python, Snowflake, and AWS. Implement CI/CD for data workflows, use developer assistance tools, and collaborate across teams to deliver reliable data solutions.
Top Skills:
AWSPythonSnowflakeSQL
What you need to know about the Kolkata Tech Scene
When considering the industries shaping India's tech scene, gaming might not immediately come to mind. However, in the last decade, increased internet usage and greater access to mobile devices have catapulted the industry to new heights, with Kolkata-based companies like Virtualinfocom, Red Apple Technologies and Digitoonz, at the forefront, driving the design and animation of new gaming titles for players.
