Design, implement, and optimize cloud-native data pipelines and ETL workflows using ADF, DBT, Python, and Snowflake. Build data models, integrate schedulers and Azure Function Apps, apply GenAI tools for productivity, ensure data governance and quality, mentor junior engineers, and collaborate with stakeholders in Agile teams.
Job Summary
Senior Data Engineer
Location
TechM Blr ITPL
Years of Experience
10 12 years
Job Summary
The Senior Data Engineer will design, develop, and optimize data solutions to support business processes and analytics within Investment Management. This role requires deep expertise in modern data engineering practices, cloud native development, and advanced data transformation techniques. The engineer will work extensively with SQL, DBT, Snowflake, Azure Data Factory (ADF), Python, and job scheduling tools, while leveraging Azure Function Apps and Gen AI tools such as Cortex AI and GitHub Copilot for automation and productivity.
Responsibilities
Regular work hours: 11.00 AM to 8.00 PM IST, with flexibility in work hours. This position allows for Hybrid work which requires the individual to be in office 2 3 days a week. Individual must be available to work US Eastern Standard Time hours as business requires.
Senior Data Engineer
Location
TechM Blr ITPL
Years of Experience
10 12 years
Job Summary
The Senior Data Engineer will design, develop, and optimize data solutions to support business processes and analytics within Investment Management. This role requires deep expertise in modern data engineering practices, cloud native development, and advanced data transformation techniques. The engineer will work extensively with SQL, DBT, Snowflake, Azure Data Factory (ADF), Python, and job scheduling tools, while leveraging Azure Function Apps and Gen AI tools such as Cortex AI and GitHub Copilot for automation and productivity.
Responsibilities
- Architect and implement scalable data pipelines and ETL workflows using ADF, DBT, and Python.
- Design and optimize data models for Snowflake and other cloud based data platforms.
- Develop and maintain complex data transformation logic using SQL, DBT, Python, Snowpark, and the Snowsight web interface.
- Integrate job schedulers with Azure Data Factory pipelines and Azure Function Apps for orchestration and automation.
- Apply Gen AI tools (GitHub Copilot Agent Skills, Snowflake Cortex AI) to accelerate development, improve code quality, and enhance productivity.
- Ensure robust data governance, security, and compliance across all solutions.
- Collaborate with cross functional teams to understand business requirements and translate them into technical solutions.
- Mentor junior engineers and provide technical leadership in data engineering best practices.
- Participate in Scrum ceremonies and contribute to continuous improvement initiatives through strong ownership and collaboration.
- 10 12 years of hands on experience in data engineering with strong coding skills.
- Proven expertise in DBT, Python, Azure Data Factory (ADF), SQL, Snowflake, and Azure Databricks.
- Expert in SQL and Python for developing and debugging scalable data pipelines.
- Experience with job schedulers and Azure Function Apps for automation.
- Strong use of infrastructure as code tools like Terraform to manage deployment pipelines.
- Design and optimize API integration in pipelines.
- Advanced understanding of dimensional modeling and data warehousing concepts.
- Strong understanding of Git version control, CI/CD pipelines, DevSecOps, and Agile methodologies.
- Excellent problem solving skills and ability to work in a fast paced environment.
- Strong communication and stakeholder management skills.
- Experience with various LLMs and Natural Language Processing (NLP).
- Experience with Azure Databricks and PySpark.
- Experience with data testing, observability, and data quality frameworks such as SODA, Great Expectations, DBT Tests.
- Bachelor of Engineering degree in Computer Science, Information Technology, or related field.
- Certifications in Azure Data Engineering, Snowflake are a plus.
- Exposure to Investment Management or Financial Services domain preferred.
Regular work hours: 11.00 AM to 8.00 PM IST, with flexibility in work hours. This position allows for Hybrid work which requires the individual to be in office 2 3 days a week. Individual must be available to work US Eastern Standard Time hours as business requires.
Similar Jobs at Navixus | Tech Mahindra
Artificial Intelligence • Natural Language Processing • Professional Services • Analytics • Consulting • Conversational AI • Generative AI
Administer and maintain Hadoop clusters and relational databases, monitor and tune performance, perform backups and recovery, manage security and user access, automate tasks with scripting, collaborate with dev teams on ETL/data modeling and document configurations and procedures.
Top Skills:
AWSAzureBashCassandraData WarehousingETLHadoopMongoDBMySQLOraclePostgresPower BIPythonTableau
What you need to know about the Kolkata Tech Scene
When considering the industries shaping India's tech scene, gaming might not immediately come to mind. However, in the last decade, increased internet usage and greater access to mobile devices have catapulted the industry to new heights, with Kolkata-based companies like Virtualinfocom, Red Apple Technologies and Digitoonz, at the forefront, driving the design and animation of new gaming titles for players.
