Design, build, and optimize large-scale ETL pipelines and distributed data applications (batch and streaming) using Spark and Scala. Ensure data quality and performance, deploy and manage workloads in cloud/container environments, troubleshoot production issues, and collaborate with product, engineering, and analytics teams to deliver scalable AdTech data solutions.
We are looking for Senior Data Engineers to join our AdTech team and build scalable, high-performance data platforms that power advertising insights and analytics. The ideal candidate will have strong experience in distributed data processing, ETL pipelines, and Big Data technologies, with hands-on expertise in Spark and Scala.
You will work on designing, developing, and optimizing large-scale data pipelines while collaborating closely with cross-functional engineering teams to build reliable, production-grade data solutions.
Key Responsibilities
- Design, develop, and maintain scalable ETL pipelines for large-scale data processing.
- Build and optimize distributed data applications using Spark and Scala.
- Develop reliable, high-performance data pipelines for batch and streaming workloads.
- Work with large datasets to ensure data quality, consistency, and performance.
- Collaborate with engineering, product, and analytics teams to deliver robust data solutions.
- Optimize data workflows for scalability, reliability, and cost efficiency.
- Deploy and manage data workloads in cloud and containerized environments.
- Troubleshoot production issues and continuously improve platform performance.
Requirements
Candidates who demonstrate:
- 5+ years of experience in Data Engineering or Big Data Engineering.
- Strong hands-on experience with Apache Spark and Scala.
- Experience building and maintaining ETL pipelines.
- Familiarity with Google Cloud Storage (GCS).
- Experience with Kubernetes (K8s).
- Strong SQL skills and understanding of distributed data processing.
- Excellent debugging, problem-solving, and performance optimization skills.
- Strong communication and collaboration skills.
Good to Have
- Experience with AWS and cloud-native data services.
- Familiarity with streaming technologies such as Kafka.
- Experience working on large-scale data platforms or AdTech systems.
- Exposure to orchestration tools such as Airflow.
Benefits
- Best-in-class salary: We hire strong talent and compensate accordingly.
- Proximity Talks: Meet and learn from designers, engineers, product leaders, and AI practitioners.
- Continuous learning: Work with a world-class team and stay close to the latest in AI, engineering, and product development.
- High-impact work: Build AI-first systems and products used at scale by global clients.
About Us
Proximity is the trusted technology, design, and consulting partner for some of the biggest Sports, Media, and Entertainment companies in the world. We’re headquartered in San Francisco and have offices in Palo Alto, Dubai, Mumbai, and Bangalore.
Since 2019, Proximity has built high-impact, scalable products used by millions of users every day. Today, we are a global team of engineers, designers, product managers, and experts solving complex problems and building cutting-edge technology at scale.
Similar Jobs
Enterprise Web • Mobile • Professional Services • Software
Build and improve production LLM agents and AI features through evaluation, experimentation, observability, prompting, context engineering, tool use, and workflow design. Develop backend services, APIs, data models, and feedback pipelines that make agent behavior reliable and measurable. Investigate performance issues, run staged rollouts and production replays, define quality standards, and partner with Product, Data Science, and Sales to deliver business outcomes with appropriate privacy, security, and human-oversight safeguards.
Top Skills:
Agentic WorkflowsAPIsBackend ServicesBraintrustClaude CodeCursorData ModelsDatadog Llm ObservabilityEvaluation HarnessesGithub CopilotLangchainLanggraphLangsmithLlm-As-JudgeLlm-Based SystemsMcp
Enterprise Web • Mobile • Professional Services • Software
Own the AI product roadmap from discovery through launch and iteration. Define model evaluation standards, failure modes, rollback plans, and agent-autonomy boundaries. Prototype and ship production AI features, influence architecture, and apply prompting and context design. Collaborate with Design, Research, Engineering, Sales, Marketing, and Customer Success while tracking quality, latency, cost, adoption, and growth metrics.
Top Skills:
AIBraintrustClaudeCodexCursorLangsmithLlmsOpenclaw
Enterprise Web • Mobile • Professional Services • Software
Build and own full-stack product features from ambiguous problems through deployment. Partner with Product and Design, make UX decisions, integrate LLM and agent capabilities into trustworthy product experiences, talk with users, instrument releases, and iterate based on usage. The role requires strong product judgment, user empathy, independent execution, and comfort using AI coding tools while collaborating with AI engineers on model behavior.
Top Skills:
Claude CodeCursorGithub CopilotLlm AgentsLlm ApisPrompting
What you need to know about the Kolkata Tech Scene
When considering the industries shaping India's tech scene, gaming might not immediately come to mind. However, in the last decade, increased internet usage and greater access to mobile devices have catapulted the industry to new heights, with Kolkata-based companies like Virtualinfocom, Red Apple Technologies and Digitoonz, at the forefront, driving the design and animation of new gaming titles for players.
