The Data Engineer will design and maintain data pipelines, build data warehouses, and ensure data quality while collaborating with data scientists and engineers.
Job Summary:
We are looking for a talented and experienced Data Engineer to join our team. As a Data Engineer, you will be responsible for designing, building, and maintaining our data pipelines and infrastructure. You will work closely with our data scientists and analysts to ensure that our data is accessible, reliable, and secure.
Responsibilities:
- Design, develop, and maintain scalable data pipelines.
- Build and manage data warehouses and data lakes.
- Implement data quality and data governance best practices.
- Work with data scientists and analysts to support their research and development projects.
- Collaborate with other engineers to build and maintain our data infrastructure.
Qualifications:
- Bachelor's degree in Computer Science, Engineering, or a related field
- 3-5 years of experience in Data Warehousing and Data Integration projects, with exposure to ETL processes, data pipelines, and data quality management.
- practical experience working within large scale data environments and demonstrate strong analytical and problem-solving capabilities.
- Hands-on experience in SAS, including development and optimization of SAS programs.
- Strong programming skills in Python, Java, or Scala
- Experience with big data technologies such as Hadoop, Spark, and Kafka
- Experience with cloud computing platforms such as AWS, Azure, or Google Cloud Platform
- Experience with database systems such as SQL, PostgreSQL and Cassandra
- Experience with data warehousing and ETL (extract, transform, load) processes
- Excellent problem-solving and analytical skills
- Strong communication and teamwork skills
Bonus Points:
- Experience with SAS, Kubernetes, or Docker
- Experience with machine learning and artificial intelligence
- Experience with cloud-native development
Similar Jobs
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Design, build, and maintain cloud-native data pipelines and ETL/ELT workflows for high-volume trade lifecycle data. Integrate DTCC sources (RTTM, Settlement Web) into a Unified Trade Record, process ISO 15022 messages, manage trade exception databases, optimize SQL Server performance, and ensure data quality, lineage, governance, and observability while collaborating with business stakeholders.
Top Skills:
AWSDockerDtcc RttmDtcc Settlement WebEksEltETLFixFxmlIbm MqIndexingIso 15022KafkaKubernetesRdsRest ApiS3SQLSQL ServerStored ProceduresUnified Trade Record (Utr)
Cloud • Security • Software • Cybersecurity • Automation
Lead design and implementation of in-product insights built on the Data Insights Platform and Knowledge Graph. Architect data ingestion, modeling, and synchronization; integrate third-party systems; design a Data Marketplace for Snowflake/Databricks/BigQuery access; publish observability via OpenTelemetry; define operational readiness across deployment types; coordinate cross-team delivery; and mentor senior/intermediate engineers.
Top Skills:
BigQueryData Insights PlatformDatabricksGoJIRAKnowledge GraphNode.jsOpentelemetryRubyServicenowSnowflakeZendesk
Cloud • Security • Software • Cybersecurity • Automation
Build fullstack features for data ingestion, modeling, knowledge graph sync, integrations, and dashboards. Contribute to APIs, AI-enabled workflows, performance and reliability goals, and collaborate with senior engineers on design and delivery.
Top Skills:
AIAPIsClickhouseGoJIRAKnowledge GraphNode.jsObservability ToolsOlapRubySdksServicenowZendesk
What you need to know about the Kolkata Tech Scene
When considering the industries shaping India's tech scene, gaming might not immediately come to mind. However, in the last decade, increased internet usage and greater access to mobile devices have catapulted the industry to new heights, with Kolkata-based companies like Virtualinfocom, Red Apple Technologies and Digitoonz, at the forefront, driving the design and animation of new gaming titles for players.
