Similar Jobs
Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
Design and maintain scalable data pipelines using Azure technologies, optimize processing solutions, and collaborate with stakeholders for data solutions.
Top Skills:
SparkAzure Data FactoryAzure Data LakeAzure DatabricksPower BIPythonScalaSQL
Information Technology
Design, develop, and implement large-scale Azure-based data engineering solutions. Build and maintain data lakes, ETL/streaming pipelines using ADF, EventHub, Databricks, Functions, and ASA. Use SQL, Python, Spark, and work with Teradata, Cassandra, MongoDB, SQL Server, Snowflake and ADLS. Implement CI/CD with GitHub, Azure DevOps and Terraform. Collaborate in an Agile/DevOps hybrid model.
Top Skills:
AdlsAzure Blob StorageAzure Data Explorer (Kql)Azure Data FactoryAzure Data LakeAzure DatabricksAzure DevopsAzure EventhubAzure FunctionsAzure Power AppsAzure Stream AnalyticsCassandraGitMongoDBPythonSnowflakeSparkSQLSQL ServerTeradataTerraformUnix
Information Technology
Design, develop, and implement large-scale data engineering solutions using Azure data stack. Build data lakes, pipelines, and analytics using SQL, Python, Spark, Databricks, ADF, EventHub, and CI/CD tools. Collaborate in Agile/DevOps environments and provide development, administration, and support across multiple data platforms.
Top Skills:
AdlsAzure Blob StorageAzure Data Explorer (Kql)Azure Data FactoryAzure Data LakeAzure DatabricksAzure DevopsAzure EventhubAzure FunctionsAzure Power AppsAzure Stream Analytics (Asa)CassandraGitMongoDBPythonSnowflakeSparkSQLSQL ServerTeradataTerraformUnix
We are seeking a Data Engineer with expertise in Palantir Foundry to build scalable, normalized data models and pipelines. This role will drive data ingestion, transformation, and normalization while enabling visualization development in Power BI. The ideal candidate will work closely with the Product Manager, Data analysts, and key stakeholders to deliver clean, structured insights.
Key Responsibilities:
- Data Modeling and Normalization:
- Develop and implement scalable data models in Palantir Foundry.
- Normalize data from ERP systems (SAP, JD Edwards) to ensure cross-client consistency.
- ETL and Data Pipelines:
- Build and automate data ingestion pipelines to reduce manual intervention.
- Develop processes for cleaning, validating, and transforming large datasets.
- Visualization Development:
- Collaborate with analysts to design reusable Power BI dashboards and views.
- Ensure seamless integration of clean, normalized data into Power BI.
- Tool Integration:
- Integrate Cap IQ and other external datasets into Palantir pipelines.
- Optimize data workflows to accelerate the creation of industry-specific templates.
- Collaboration and Scalability:
- Work with the Product Manager and Data Analysts to align normalization rules and requirements, enabling modular and reusable data solutions.
- 7-10 years in data engineering
- Strong software engineering fundamentals to build high-quality pipelines
- Experience with Palantir Foundry (or similar big data platforms).
- Expertise in building ETL pipelines and automating data ingestion processes.
- Proficiency in data wrangling, normalization, and modeling.
- Hands-on experience with Power BI for creating dashboards and integrating data models.
- Strong SQL skills for data transformation and querying.
- Solid understanding of ERP data systems (SAP, JD Edwards) and attributes.
Nice-to-Have:
- Familiarity with Cap IQ APIs and integration processes.
What you need to know about the Kolkata Tech Scene
When considering the industries shaping India's tech scene, gaming might not immediately come to mind. However, in the last decade, increased internet usage and greater access to mobile devices have catapulted the industry to new heights, with Kolkata-based companies like Virtualinfocom, Red Apple Technologies and Digitoonz, at the forefront, driving the design and animation of new gaming titles for players.
