The Technical Lead will develop and maintain ELT/ETL pipelines, focusing on data quality and schema enforcement, while utilizing Databricks and PySpark.
Data Engineer - Databricks,Spark / PySpark,Kafka
• Write efficient Pyspark code and libraries for Databricks notebooks using best practices.
• Build and maintain ELT/ETL pipelines using Medallion Architecture.
• Implement batch pipelines using Databricks,Spark / PySpark,Kafka
• Ensure data quality, schema enforcement, and lineage.
• Support ingestion of healthcare data sources with incremental and CDC logic
• Maintaining the code base, code deployment and infra setup for data pipelines .
• Experience in MDM, Data Mart
• Exposure to incentive planning and compensation.
• Experience in healthcare domain
Top Skills
Databricks
Kafka
Pyspark
Spark
Similar Jobs
Insurance
The Technical Lead will oversee global Operations and Technology teams, focusing on integrating legacy systems with modern applications, cloud digital transformation, and leading operational support for applications and infrastructure.
Top Skills:
Apache CamelAppdynamicsAzureAzure DevopsDockerDynatraceElastic ElkGitlabsGrafanaJenkinsKafkaKubernetesOpensearchPrometheusSplunkSpring Boot
Insurance
The role involves designing and maintaining reporting and data integration solutions, developing ETL pipelines using SSIS, and creating interactive reports with SSRS.
Top Skills:
Azure Data FactoryGitPower BISql Server 2019/2022+SsisSsrsT-SqlTfs
Insurance
The Technical Lead will guide engineering teams, develop enterprise software solutions, ensure compliance with best practices, and mentor team members while working in an Agile environment.
Top Skills:
AngularAzureBootstrapCSSDotnetGraphQLHTMLJavaScriptReactRestSoapTypescript
What you need to know about the Kolkata Tech Scene
When considering the industries shaping India's tech scene, gaming might not immediately come to mind. However, in the last decade, increased internet usage and greater access to mobile devices have catapulted the industry to new heights, with Kolkata-based companies like Virtualinfocom, Red Apple Technologies and Digitoonz, at the forefront, driving the design and animation of new gaming titles for players.
