Yadimen Consulting Limited Logo

Yadimen Consulting Limited

Data Engineer — PySpark + AWS Glue

Posted 25 Days Ago
Be an Early Applicant
In-Office
Pune, Mahārāshtra
Senior level
In-Office
Pune, Mahārāshtra
Senior level
Design, develop, optimize, and maintain scalable ETL pipelines using PySpark and AWS Glue. Build data ingestion and transformation workflows, orchestrate processes with AWS Step Functions, and develop Python-based AWS Lambda functions. Support AWS data lake architectures, analytical use cases, monitoring, reliability, and performance optimization. The role also involves SQL and may include Java microservices, REST APIs, and backend integrations.
The summary above was generated by AI

Data Engineer — PySpark + AWS Glue

ETL & Data Pipelines | 2 Openings

 

 

Location: Remote (UK)

Employment Type: Contract / Permanent

Experience Level: 4–8 Years

Openings: 2

 

About the Role

We are looking for a skilled Data Engineer with solid hands-on experience in PySpark, AWS Glue, and ETL development to build and maintain scalable, production-grade data pipelines on AWS. You will be part of a delivery-focused team working on complex data engineering challenges, contributing to data lake architecture and end-to-end pipeline development.



Requirements

Key Responsibilities

       Design, develop, and maintain scalable ETL pipelines using PySpark and AWS Glue

       Build and optimise data ingestion, transformation, and loading workflows

       Orchestrate data workflows using AWS Step Functions

       Develop serverless functions using AWS Lambda (Python)

       Work with data lake architectures on AWS to support analytical use cases

       Ensure pipeline reliability, monitoring, and performance optimisation

 

Required Skills & Experience

       Strong hands-on experience with PySpark and AWS Glue

       Proven track record in ETL pipeline development and optimisation

       Experience orchestrating workflows with AWS Step Functions

       Proficiency in serverless development using AWS Lambda (Python)

       Good SQL skills and a solid understanding of data processing principles

 

Nice to Have

       Exposure to Java-based microservices

       Understanding of REST APIs and backend service integrations

       Experience working with AWS-based data lakes



Similar Jobs

An Hour Ago
Hybrid
Junior
Junior
Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
Executes procure-to-pay activities, including requisition processing, purchase order creation and maintenance, vendor onboarding, quote comparison, routine negotiations, and invoice discrepancy resolution. Manages procurement queues and SLAs, maintains audit-ready records, supports contract reviews and compliance, and provides reporting on open POs, aging requisitions, and cycle times. Partners with stakeholders, suppliers, Accounts Payable, Legal, and senior procurement team members while identifying process improvements and using AI to streamline workflows.
Top Skills: Ai ToolsAribaCoupaGoogle SheetsJIRAExcelNetSuiteOracleSAPServicenowWorkday
Yesterday
Remote or Hybrid
India
Junior
Junior
Aerospace • Artificial Intelligence • Cloud • Machine Learning • Software • Cybersecurity • Defense
Conduct third-party due diligence and compliance screenings, coordinate escalations, monitor screening systems, and ensure adherence to laws and company policies. The role also develops ethics and compliance training, collaborates cross-functionally, and drives data analytics and insights for the Third Party Risk and Digital Compliance team.
Yesterday
Remote or Hybrid
India
Junior
Junior
AdTech • Big Data • Digital Media • Software
Designs and reviews security architecture, performs threat modeling and cloud assessments, responds to incidents, manages security tools, and advises engineering teams on secure development. Conducts code and vulnerability reviews using OWASP and MITRE ATLAS, evaluates AI and LLM security risks, and supports security monitoring, risk assessment, data protection, and compliance across applications, infrastructure, and endpoints.
Top Skills: AWSC++DastGoogle SecopsHmacIso 27001IspmJavaLinuxmacOSMitre AtlasNist CsfOauthOwaspOwasp Top 10 For LlmsPythonSAMLSastScalaSoc 1Soc 2SoxSplunkTenable CloudWazuhWindows

What you need to know about the Kolkata Tech Scene

When considering the industries shaping India's tech scene, gaming might not immediately come to mind. However, in the last decade, increased internet usage and greater access to mobile devices have catapulted the industry to new heights, with Kolkata-based companies like Virtualinfocom, Red Apple Technologies and Digitoonz, at the forefront, driving the design and animation of new gaming titles for players.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account