The Azure Data Engineer will implement big data pipelines using Azure technologies, collaborate with stakeholders, and utilize tools like Data Factory and Databricks.
This is a remote position.
Description :
- At least 2 years hands on experience working with an Agile data engineering team working on big data pipelines using Azure in a commercial environment.
- Dealing with senior stakeholders/leadership
- Understanding of Azure data security and encryption best practices. [ADFS/ACLs]
- Data Bricks - experience writing in and using data bricks Using Python to transform, manipulate data.
- Data Factory - experience using data factory in an enterprise solution to build data pipelines. Experience calling rest APIs.
- Synapse/data warehouse - experience using synapse/data warehouse to present data securely and to build & manage data models.
- Microsoft SQL server - We- d expect the candidate to have come from a SQL/Data background and progressed into Azure
- PowerBI - Experience with this is preferred
Additionally :
- Experience using GIT as a source control system
- Understanding of DevOps concepts and application
- Understanding of Azure Cloud costs/management and running platforms efficiently
Requirements
- 3 to 6 years of experience.
- Must Have: Azure services like Azure Data Factory, Databricks, Azure Data lakes, Synapse/DW. and SQL
- Good to have: Python, PowerBI, Git
Similar Jobs
Cloud • Information Technology • Business Intelligence • Consulting
Design, build, and deliver production-ready Data & AI solutions on Microsoft Azure. Develop and maintain data pipelines and lakehouse architectures using Fabric, Databricks, and Azure services; implement CI/CD, optimize production workloads, and collaborate with architects, project teams, and clients.
Top Skills:
AzureAzure Data FactoryAzure Data Lake StorageAzure DevopsAzure SqlAzure StorageCopilot StudioDatabricksGitMicrosoft CopilotMicrosoft FabricPysparkPythonSQLT-Sql
Information Technology • Professional Services • Software • Consulting
Design, build and maintain secure, high-quality data pipelines on Azure (Data Factory, Databricks, Synapse). Cleanse, transform and optimise large event-based datasets, implement data quality/observability, develop ETL scripts, stored procedures, and CI/CD pipelines, and participate in testing and troubleshooting.
Top Skills:
SparkAzureAzure Data FactoryAzure DatabricksAzure SynapseCi/CdETLPysparkPythonSparkSQL Server
Software
Design, build, and maintain batch and real-time ETL/ELT data pipelines and workflows. Automate ingestion, aggregation and logging; ensure data accuracy, security, and compliance. Implement data warehouse and lake solutions, mentor junior engineers, and collaborate with stakeholders to meet business requirements.
Top Skills:
AdfAdls Gen2AirflowAthenaAWSAws QuicksightAzureAzure Sql DbAzure SynapseBigQueryCassandraData LakesData WarehousesEc2EmrFivetranGCPGlueGlue EtlHadoopHiveIamJavaLambdaMapreduceMatillionMongoDBNeo4JNumpyPandasPower BIPythonRdsRedisRedshiftS3ScalaSnowflakeSparkSQLStitchTableau
What you need to know about the Kolkata Tech Scene
When considering the industries shaping India's tech scene, gaming might not immediately come to mind. However, in the last decade, increased internet usage and greater access to mobile devices have catapulted the industry to new heights, with Kolkata-based companies like Virtualinfocom, Red Apple Technologies and Digitoonz, at the forefront, driving the design and animation of new gaming titles for players.
