Designs and optimizes scalable data ingestion pipelines for product telemetry and enterprise business data. Builds automated pipelines using Fivetran, Python, APIs, S3 lakehouse architectures, Kafka, Spark, and Snowflake. Applies advanced SQL, incremental loading, CDC, micro-batching, and rate-limit handling to support reliable, analytics-ready datasets. Implements CI/CD and DevOps practices, monitors pipeline performance, and partners with analysts and platform teams to rapidly deliver data solutions.
Your role at Dynatrace
We are seeking an experienced Data Engineer to design, build, and optimize scalable data ingestion pipelines for product telemetry and enterprise business applications data. This role operates in a fast‑paced environment, rapidly prototyping data pipelines to enable data exploration, analytics, and downstream consumption.
Responsibilities
- Design and implement modern data ingestion pipelines for business systems data using Fivetran, Python and Bulk APIs, S3-based lakehouse architectures, and streaming ingestion (Kafka, Spark)
- Build end-to-end automated pipelines from source systems to Snowflake, ensuring reliability, scalability, and observability
- Apply advanced SQL in Snowflake for transformation, standardization, and preparation of analytics-ready datasets
- Optimize ingestion performance for large-scale, high-volume datasets using API rate-limit handling, micro-batching, and incremental loading / CDC
- Implement DevOps and CI/CD practices for data pipelines, including version control and deployment automation
- Partner with analysts, SMEs, and platform teams to rapidly prototype pipelines supporting exploration and evolving requirements
- Monitor, troubleshoot, and optimize ingestion jobs to ensure data freshness, accuracy, and performance
- 6+ years of hands-on experience in data engineering and ingestion automation in cloud or hybrid environments
- Proven experience ingesting business application data (CRM, ERP, financial, consumption, renewals systems)
- Advanced proficiency with Python and Fivetran (or similar ETL/ELT tool) for pipelines, automation, orchestration, and validation
- Expertise with Kafka, Spark, and S3 / lakehouse-style storage architectures
- Strong proficiency with Snowflake, ingestion, transformation, and performance optimization
- Deep understanding of ingestion optimization, rate limits, micro-batching, incremental loads, and CDC
- Experience applying DevOps / CI/CD practices to data pipelines
- Ability to work independently and deliver solutions rapidly in a fast-paced, iterative environment
Desirable:
- Experience with additional open-source ingestion frameworks and connectors
- Exposure to event-driven or telemetry-heavy architectures at scale
- Dynatrace is a leader in unified observability and security.
- We provide a culture of excellence with competitive compensation packages designed to recognize and reward performance.
- Our employees work with the largest cloud providers, including AWS, Microsoft, and Google Cloud, and other leading partners worldwide to create strategic alliances.
- You'll get to work at the forefront of innovation with Dynatrace Intelligence, the industry's first agentic operations system. Bringing together deterministic and agentic AI, it helps teams understand what's happening, why it matters, and what to do next, automatically.
- Over 50% of the Fortune 100 companies are current customers of Dynatrace
Similar Jobs at Dynatrace
Artificial Intelligence • Big Data • Cloud • Information Technology • Software • Big Data Analytics • Automation
Designs and deploys production AI assistants, copilots, agents, RAG systems, and intelligent automations. Integrates LLMs with Snowflake, AWS, enterprise applications, and workflow platforms to automate business processes. Builds backend services and serverless workflows, implements observability and responsible AI controls, and scales validated prototypes into reliable production solutions. Partners with technical and business teams to improve decision-making, efficiency, quality, and cost performance.
Top Skills:
AirflowAmazon Api GatewayAmazon BedrockAmazon EventbridgeAmazon S3Amazon SagemakerAnthropicAws LambdaAws Step FunctionsAzure OpenaiDbtGeminiGoMcpMicrosoft Power AutomateMlflowN8NOauthOpenaiPythonRbacSalesforceServicenowSnowflakeSnowflake Cortex SearchSnowflake Ml RegistrySnowparkSsoTypescriptUipathWeights & BiasesWorkato
Artificial Intelligence • Big Data • Cloud • Information Technology • Software • Big Data Analytics • Automation
Manages APAC payroll operations through an outsourced CloudPay model, ensuring accurate, timely, and compliant employee payments. Oversees payroll processing, reconciliations, statutory reporting, controls, vendor relationships, KPIs, SLAs, and payroll sign-off. Leads and develops the payroll team, coordinates projects, resolves issues, supports continuous improvement, and partners with P&C, Finance, HRIS, senior leadership, and external providers.
Top Skills:
CloudpayExcelSfec HcmSox Compliance
What you need to know about the Kolkata Tech Scene
When considering the industries shaping India's tech scene, gaming might not immediately come to mind. However, in the last decade, increased internet usage and greater access to mobile devices have catapulted the industry to new heights, with Kolkata-based companies like Virtualinfocom, Red Apple Technologies and Digitoonz, at the forefront, driving the design and animation of new gaming titles for players.
