While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth.
If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi!
Role: Senior Data Engineer (Databricks)
Experience Level: 3 to 6 Years
Work location: Mumbai, Bengaluru, Trivandrum
Role & Responsibilities:
Design and implement data solutions (ETL pipelines, Processing and Transformation logic) using Snowflake as a key platform.
Design Virtual warehouses which are optimized for performance and cost (auto scaling, idle time etc)
Write SQL , stored procedures to implement business logic, other critical requirements like encryption, data masking etc
Implement solutions with features like Snowpipe, tasks, streams,dynamic tables etc
Design the solutions by leveraging key concepts around the architecture of Snowflake, its data distribution and partitioning mechanism, data caching etc.
Identify queries which take more time , look at the logs etc and provide solutions to optimize those queries
Must have skills:
- 3+ years’ experience of Hands-on in data structures, distributed systems, Spark, SQL, PySpark, NoSQL Databases
Strong software development skills in at least one of: Python, Pyspark or Scala.
Develop and maintain scalable & modular data pipelines using databricks & Apache Spark
Experience building and supporting large-scale systems in a production environment
Experience working on either of the cloud storages - AWS S3, Azure data lake, GCS buckets
Exposure to the ELT tools offered by the cloud platforms like, ADF, AWS Glue, Google dataflow
Integrate Databricks with other cloud services like AWS, Azure or Google Cloud
Implement and optimize Spark jobs, data transformations, and data processing workflows in Databricks
Managing and configuring databricks environments, including clusters and notebooks
Experience working with different file formats - parquet, databricks delta lake
Experience working with data governance on databricks, implementing unity catalog, delta share
If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us!
