Senior Big Data Engineer responsible for Azure Databricks development and end-to-end data engineering. Must be hands-on with PySpark, write advanced SQL, work with data ingestion, preparation, integration, and operationalize pipelines across Data Lakes, Warehouses and Marts. Own coding, testing, debugging, deployment; DWH experience required. Strong communication skills.
Overall 6.5 - 10 yrs of IT experience preferably in cloud· Min 5 years in Azure Databricks on development projects· Should be 100% hands on in Pyspark coding· Should have strong SQL expertise in writing advanced/complex SQL queries· DWH experience is a must for this role· Experience in programming using Python is an advantage · Experience in data ingestion, preparation, integration, and operationalization techniques in optimally addressing the data requirements· Should be able to understand system architecture which involves Data Lakes, Data Warehouses and Data Marts· Experience to own end-to-end development, including coding, testing, debugging and deploymentExcellent communication is required for this role
Similar Jobs
Information Technology • Consulting
Design, implement, and automate Snowflake data ingestion and DBT transformation pipelines (Data Vault 2.0). Troubleshoot data operations, implement data quality monitoring, optimize platform performance, and deploy code via CI/CD while collaborating with the team.
Top Skills:
Aws EmrAws GlueAws LambdaAzureCi/CdData Vault 2.0DbtDevOpsFivetranPythonSnowflakeSnowpipe
Information Technology • Consulting
Hands-on Big Data Engineer responsible for PySpark development on Azure Databricks, advanced SQL, and data warehouse/DWH work. Own data ingestion, preparation, integration, operationalization, and end-to-end development (coding, testing, debugging, deployment). Understand data lakes, data warehouses, and data marts. Strong communication required.
Top Skills:
AzureAzure DatabricksData LakeData MartData WarehousePysparkPythonSQL
Information Technology • Consulting
The Big Data Engineer requires 3-6 years of hands-on experience with PySpark, ADF, ADB, and SQL, focusing on big data technologies.
Top Skills:
AdbAdfPysparkSQL
What you need to know about the Kolkata Tech Scene
When considering the industries shaping India's tech scene, gaming might not immediately come to mind. However, in the last decade, increased internet usage and greater access to mobile devices have catapulted the industry to new heights, with Kolkata-based companies like Virtualinfocom, Red Apple Technologies and Digitoonz, at the forefront, driving the design and animation of new gaming titles for players.
