Design, develop, and maintain scalable cloud-based data pipelines and ETL/ELT processes. Build and optimize data solutions on GCP (BigQuery, Dataflow), ensure data quality, governance, monitoring, and high availability. Develop backend data processing services in Java, support BI, analytics, and ML teams, troubleshoot pipeline issues, and contribute to data platform architecture and modernization.
This role is for one of the Weekday's clients
Salary range: Rs 1200000 - Rs 2500000 (ie INR 12-25 LPA)
Experience: 7+ yrs
Location: Pune, Bangalore, Chennai, Coimbatore
Job Type: full-time
We are looking for a skilled Data Engineer who is passionate about building scalable data platforms and enabling data-driven decision-making. This role involves designing, developing, and maintaining robust data pipelines, processing large-scale datasets, and ensuring the availability, reliability, and performance of data systems.
The ideal candidate will have strong expertise in cloud-based data engineering, backend development, and modern data processing technologies. You will work closely with cross-functional teams to create efficient data solutions that support analytics, reporting, machine learning, and business intelligence initiatives.
RequirementsKey Responsibilities
- Design, develop, and maintain scalable data pipelines for processing structured and unstructured data.
- Build and optimize cloud-based data solutions using modern data engineering best practices.
- Develop and manage data ingestion, transformation, validation, and orchestration workflows.
- Create reliable ETL/ELT pipelines to support analytics, reporting, and operational use cases.
- Work with large datasets to ensure data quality, consistency, and accessibility across systems.
- Develop and optimize backend services and data processing applications using Java.
- Implement data governance, monitoring, and performance optimization practices.
- Collaborate with product, analytics, and engineering teams to understand data requirements and deliver scalable solutions.
- Troubleshoot data pipeline issues and proactively identify opportunities for improvement.
- Ensure high availability, security, and scalability of data infrastructure.
- Participate in architecture discussions and contribute to data platform modernization initiatives.
- Support business intelligence, advanced analytics, and machine learning teams with reliable data assets.
- Strong experience in Data Engineering and building scalable data processing systems.
- Proficiency in Java and cloud-native data engineering practices.
- Hands-on experience with Google Cloud Platform (GCP) and its data ecosystem.
- Strong understanding of data warehousing, data modeling, and distributed data processing concepts.
- Experience building and maintaining high-performance ETL/ELT pipelines.
- Familiarity with BigQuery, Dataflow, and other cloud-based data processing technologies is highly desirable.
- Strong SQL skills and experience working with large-scale datasets.
- Understanding of data architecture, data governance, and data quality frameworks.
- Experience with workflow orchestration, monitoring, and automation tools.
- Strong analytical thinking and problem-solving abilities.
- Ability to collaborate effectively with cross-functional stakeholders and communicate technical concepts clearly.
- Self-driven mindset with a focus on ownership, scalability, and continuous improvement.
Similar Jobs
Blockchain • Fintech • Payments • Consulting • Cryptocurrency • Cybersecurity • Quantum Computing
Lead design and build cloud-native batch and real-time data platforms. Develop scalable ETL/ELT and streaming pipelines (Spark, Kafka, Flink), ensure data quality, governance, observability, security, cost optimization, and drive platform engineering excellence and team mentorship.
Top Skills:
AdlsAirflowApache FlinkApache Spark (CoreArm TemplatesAvroAWSAzureBigQueryCloudFormationDatabricksDockerDynatraceEmrGatlingGCPGcsGdprGithub ActionsGoIamJava (Jdk 8+)JenkinsJmeterJunitKafkaKubernetesOrcParquetPci DssPrometheusPythonRedshiftS3SnowflakeSpark StreamingSplunkSQLSQLStructured Streaming)Terraform
Blockchain • Fintech • Payments • Consulting • Cryptocurrency • Cybersecurity • Quantum Computing
Build, operate, and own end-to-end Databricks-based data pipelines and platform configuration. Collaborate with engineers and data scientists to deliver scalable, reliable data solutions, ensure production readiness, apply software engineering best practices, and drive operational excellence and cost/performance optimization.
Top Skills:
Ai AgentsAmazon S3SparkAWSCi/CdDatabricksDockerDynamoDBElasticacheGrafanaJavaKafkaKinesisKubernetesNoSQLPrometheusPythonRedisScalaSQLTerraform
Blockchain • Fintech • Payments • Consulting • Cryptocurrency • Cybersecurity • Quantum Computing
Design, build, and scale cloud-native data and BI solutions on Databricks and Snowflake. Develop high-performance data pipelines, enable multi-cloud interoperability (AWS/Azure), implement governance and secure access patterns, maintain CI/CD and infrastructure-as-code for platform provisioning, and lead technical design while mentoring engineers.
Top Skills:
Adls Gen2AksApache IcebergAWSAzureAzure DatabricksCi/CdDatabricksDelta LakeEmrEnterprise Identity IntegrationGlueIamInfrastructure As CodeRbacS3Service PrincipalsSnowflake
What you need to know about the Kolkata Tech Scene
When considering the industries shaping India's tech scene, gaming might not immediately come to mind. However, in the last decade, increased internet usage and greater access to mobile devices have catapulted the industry to new heights, with Kolkata-based companies like Virtualinfocom, Red Apple Technologies and Digitoonz, at the forefront, driving the design and animation of new gaming titles for players.
