Design, build, and maintain scalable ETL/ELT data pipelines and platforms on Azure Databricks. Ingest and transform structured and unstructured data using PySpark and Spark SQL, manage ADLS Gen2 storage, ensure data quality and performance, and collaborate with data science and BI teams to support analytics and reporting.
Company Description
Blend is a premier AI services provider, committed to co-creating meaningful impact for its clients through the power of data science, AI, technology, and people. We help organisations solve complex business challenges by combining deep domain understanding with modern data and AI capabilities.
Our teams work across strategy, analytics, engineering, and product delivery to create scalable, high-value solutions that improve decision-making, efficiency, and growth.
Job DescriptionWe are looking for a skilled Data Engineer with 5+ years of experience to design, build, and maintain scalable data pipelines and platforms using Azure and Databricks. The ideal candidate will work closely with analytics, data science, and business teams to deliver reliable, high-performance data solutions that enable data-driven decision-making.
Key Responsibilities
- Design, develop, and optimize end-to-end data pipelines using Azure Databricks
- Build and maintain scalable ETL/ELT frameworks for structured and unstructured data
- Ingest data from multiple sources including APIs, databases, files, and streaming systems
- Work with Azure Data Lake Storage (ADLS Gen2) for efficient data storage and retrieval
- Develop data transformation logic using PySpark and Spark SQL
- Ensure data quality, reliability, and performance across pipelines
- Implement data modeling concepts to support analytics and reporting use cases
- Collaborate with Data Scientists, BI teams, and business stakeholders to understand data requirements
- Monitor, troubleshoot, and optimize Databricks jobs for performance and cost efficiency
- Apply best practices around security, governance, and compliance within Azure
- Participate in code reviews, documentation, and continuous improvement initiatives
- 5+ years of hands-on experience as a Data Engineer
- Strong experience with Azure Databricks
- Proficiency in PySpark, Spark SQL, and Python
- Experience working with Azure Data Lake Storage (ADLS Gen2)
- Solid understanding of data warehousing and data modeling concepts
- Experience building batch and/or streaming data pipelines
- Familiarity with Azure Data Factory or similar orchestration tools
- Strong SQL skills and experience with relational databases
- Experience using Git or similar version control systems
- Strong analytical, problem-solving, and communication skills
- Experience with Delta Lake
- Knowledge of streaming technologies (Kafka, Azure Event Hubs, Spark Structured Streaming)
- Exposure to CI/CD pipelines using Azure DevOps
- Experience working in Agile/Scrum environments
- Understanding of cloud cost optimization strategies
Similar Jobs
Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
Design and maintain scalable data pipelines using Azure technologies, optimize processing solutions, and collaborate with stakeholders for data solutions.
Top Skills:
SparkAzure Data FactoryAzure Data LakeAzure DatabricksPower BIPythonScalaSQL
Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
Design, build, and maintain data pipelines using Azure technologies and Databricks. Collaborate with stakeholders on data solutions and implement data quality measures.
Top Skills:
SparkAzure Data FactoryAzure Data LakeAzure DatabricksAzure SynapsePower BIPythonScalaSQL
Digital Media • eCommerce • Gaming • Mobile • News + Entertainment
Design, deliver, and operationalize LLM- and generative AI-driven platform services. Build scalable model-serving, orchestration, MLOps, observability, data lineage, and evaluation frameworks to productionize multi-step automated workflows.
Top Skills:
AWSData PipelinesGCPGenerative AiInfrastructure As CodeKubernetesLlmsMlopsMonitoringNode.jsObservabilityPythonRagRetrieval/EmbeddingStreamingTypescriptWorkflow Orchestration
What you need to know about the Kolkata Tech Scene
When considering the industries shaping India's tech scene, gaming might not immediately come to mind. However, in the last decade, increased internet usage and greater access to mobile devices have catapulted the industry to new heights, with Kolkata-based companies like Virtualinfocom, Red Apple Technologies and Digitoonz, at the forefront, driving the design and animation of new gaming titles for players.
