Responsible for managing a team of Software Development Engineers in software planning and design, optimizing performance, and collaborating on data pipeline development and quality assurance.
Comcast brings together the best in media and technology. We drive innovation to create the world's best entertainment and online experiences. As a Fortune 50 leader, we set the pace in a variety of innovative and fascinating businesses and create career opportunities across a wide range of locations and disciplines. We are at the forefront of change and move at an amazing pace, thanks to our remarkable people, who bring cutting-edge products and services to life for millions of customers every day. If you share in our passion for teamwork, our vision to revolutionize industries and our goal to lead the future in media and technology, we want you to fast-forward your career at Comcast.
Job Summary
Responsible for leading a team of Software Development Engineers in the planning and designing of software and web applications. Manages projects from design to testing, including new programs, enhancements and modifications. Collaborates with Quality Assurance team to conduct testing. Has overall responsibility for developing/administering performance standards for organizational unit. Manages team which may include exempt and non-exempt employees. Provides subject matter guidance to employees as required. Develops processes and procedures to drive department efficiencies and assists in the development and meeting of departmental budget.
Job Description
Tech stacks:
Expertise in Pyspark, Spark, Scala Or Java
We are seeking an experienced Data Engineering Manager to design, develop, and optimize large-scale streaming systems using Apache Spark. This role involves building high-performance data pipelines, and collaborating with cross-functional teams to enable advanced analytics and machine learning.
Key Responsibilities
Required Skills & Qualifications
Nice to have
#CIECTN25
Comcast is an equal opportunity workplace. We will consider all qualified applicants for employment without regard to race, color, religion, age, sex, sexual orientation, gender identity, national origin, disability, veteran status, genetic information, or any other basis protected by applicable law.
Base pay is one part of the Total Rewards that Comcast provides to compensate and recognize employees for their work. Most sales positions are eligible for a Commission under the terms of an applicable plan, while most non-sales positions are eligible for a Bonus. Additionally, Comcast provides best-in-class Benefits to eligible employees. We believe that benefits should connect you to the support you need when it matters most, and should help you care for those who matter most. That's why we provide an array of options, expert guidance and always-on tools, that are personalized to meet the needs of your reality - to help support you physically, financially and emotionally through the big milestones and in your everyday life. Please visit the compensation and benefits summary on our careers site for more details.
Education
Bachelor's Degree
While possessing the stated degree is preferred, Comcast also may consider applicants who hold some combination of coursework and experience, or who have extensive related professional experience.
Relevant Work Experience
10 Years +
Job Summary
Responsible for leading a team of Software Development Engineers in the planning and designing of software and web applications. Manages projects from design to testing, including new programs, enhancements and modifications. Collaborates with Quality Assurance team to conduct testing. Has overall responsibility for developing/administering performance standards for organizational unit. Manages team which may include exempt and non-exempt employees. Provides subject matter guidance to employees as required. Develops processes and procedures to drive department efficiencies and assists in the development and meeting of departmental budget.
Job Description
Tech stacks:
Expertise in Pyspark, Spark, Scala Or Java
We are seeking an experienced Data Engineering Manager to design, develop, and optimize large-scale streaming systems using Apache Spark. This role involves building high-performance data pipelines, and collaborating with cross-functional teams to enable advanced analytics and machine learning.
Key Responsibilities
- Data Pipeline Development:
- Design and implement scalable streaming jobs in Cloud and on-prem
- Optimize Spark jobs for performance and cost efficiency.
- Dev Ops:
- Develop and maintain automated tooling for operational montoring and maintenance of complex pipelines
- Implement best practices for distributed data processing and storage.
- Collaboration:
- Work closely with data scientists, analysts, and business stakeholders to deliver reliable datasets.
- Data Quality & Governance:
- Implement data validation, monitoring, and lineage tracking.
- Ensure compliance with security and privacy standards.
- Continuous Improvement:
- Research and adopt new technologies to improve Spark-based data engineering processes.
Required Skills & Qualifications
- Technical Expertise:
- Strong proficiency in Spark/Scala
- Development Experience, familiarity with code repos, CI/CD
- Programming & Tools:
- Proficiency in Scala or Java.
- Cloud & Infrastructure:
- Hands-on experience with cloud platforms (AWS EMR, Azure Databricks, GCP Dataproc).
- Familiarity with CI/CD pipelines and Infrastructure as Code (Terraform, CloudFormation).
- Experience:
- 13+ years in engineering roles, with at least 6 years focused on Spark/Scala.
- Education:
- Bachelor's or Master's degree in Computer Science, Engineering, or related field.
Nice to have
- Experience with real-time streaming (Kafka, Spark Streaming).
- Dockers / Kubernetes Experience
- Databricks Experience
- Knowledge of performance tuning and troubleshooting Spark jobs.
- Certifications in cloud platforms or big data technologies.
- Strong problem-solving and communication skills.
#CIECTN25
Comcast is an equal opportunity workplace. We will consider all qualified applicants for employment without regard to race, color, religion, age, sex, sexual orientation, gender identity, national origin, disability, veteran status, genetic information, or any other basis protected by applicable law.
Base pay is one part of the Total Rewards that Comcast provides to compensate and recognize employees for their work. Most sales positions are eligible for a Commission under the terms of an applicable plan, while most non-sales positions are eligible for a Bonus. Additionally, Comcast provides best-in-class Benefits to eligible employees. We believe that benefits should connect you to the support you need when it matters most, and should help you care for those who matter most. That's why we provide an array of options, expert guidance and always-on tools, that are personalized to meet the needs of your reality - to help support you physically, financially and emotionally through the big milestones and in your everyday life. Please visit the compensation and benefits summary on our careers site for more details.
Education
Bachelor's Degree
While possessing the stated degree is preferred, Comcast also may consider applicants who hold some combination of coursework and experience, or who have extensive related professional experience.
Relevant Work Experience
10 Years +
Similar Jobs at Comcast
Digital Media • Information Technology • News + Entertainment
Design, build, and optimize CI/CD pipelines, automate deployments and infrastructure, monitor application performance, and improve collaboration between development and operations to create stable, scalable systems.
Digital Media • Information Technology • News + Entertainment
Supervise and develop an ad operations team (10+), ensure ad operations deadlines and policies are met, resolve escalations with sales, analyze verification and revenue issues, revise procedures, coach staff, and support special projects to scale operations.
Digital Media • Information Technology • News + Entertainment
Triage and root-cause analysis of field issues on RDK-based CPE devices using Linux/C/C++. Analyze device logs and machine metrics (Elastic, Splunk, Datadog), reproduce defects, provide deep source-code analysis, guide development teams, design automation for detection/triage, and lead technical activities and team improvements.
Top Skills:
CC++DatadogElasticLinuxNetworkingRdkvSplunkWifi
What you need to know about the Kolkata Tech Scene
When considering the industries shaping India's tech scene, gaming might not immediately come to mind. However, in the last decade, increased internet usage and greater access to mobile devices have catapulted the industry to new heights, with Kolkata-based companies like Virtualinfocom, Red Apple Technologies and Digitoonz, at the forefront, driving the design and animation of new gaming titles for players.
