Azure Data Engineer (ADF + Strong RDBMS and SQL)

This is a remote position.

Azure Data Engineer

We are looking for a skilled and experienced Azure Data Engineer with expertise in ADF (Azure Data Factory), strong RDBMS and SQL skills, and familiarity with Informatica PWC and Oracle PL/SQL. As an Azure Data Engineer, you will be responsible for understanding and converting Informatica PWC and Oracle PL/SQL code into ADF and Azure SQL. Additionally, you will work on Azure and Databricks "Lakehouse" implementation, translating business requirements into technical solutions for data curation and consumption.

Responsibilities:

1. Data Curation and Transformation: Translate business requirements into technical solutions for data curation and transformation, specifically using Databricks Lakehouse from a modeling and ELT perspective.
   

2. Azure Data Engineering: Develop data pipelines and orchestration using Azure Data Factory and other Azure Modern Data Warehouse platform components.
   

3. SQL and Pyspark: Utilize SQL programming and Pyspark for data manipulation and processing tasks.
   

4. Azure Services: Work with Azure services such as ADLS (Azure Data Lake Storage), ADX (Azure Data Explorer), and Databricks.
   

5. Data Migration: Experience in SQL Server migration to the cloud and handling structured and unstructured datasets.
   

6. DevOps and CI/CD: Good knowledge of DevOps tools and processes, and experience in automating data pipelines through a CI/CD delivery methodology.
   

7. Databricks Expertise: In-depth hands-on implementation knowledge on Azure Databricks, Databricks Delta Lake, and managing Delta Tables.
   

8. Data Modeling: Understand and implement data modeling for the data lake.
   

9. Data Transfer and Oracle PL/SQL: Hands-on experience in using secure file transfer tools like Kiteworks, and Oracle PL/SQL knowledge will be a plus.
   

10. Effort Estimation: Provide effort estimation for new projects related to data engineering.
    

11. Complex Design and Agile Environment: Drive complex design and development efforts in an agile environment.
    

Qualifications:

1. Minimum of 5-7 years of experience working on Azure and Databricks "Lakehouse" implementation.
   

2. Strong experience in ADF (Azure Data Factory) and SQL as primary skills.
   

3. Familiarity with Informatica PWC and Oracle PL/SQL.
   

4. Experience in SQL programming and Pyspark.
   

5. 2 or more years of experience as a Data Engineer, specifically in the Azure Modern Data Warehouse platform.
   

6. Good understanding of data modeling and data lake concepts.
   

7. Expertise in Databricks, ADLS, ADX, and other Azure services.
   

8. Prior experience in SQL Server migration to the cloud.
   

9. Knowledge of DevOps tools and processes.