The Big Data Engineer will support existing applications and work on new design and implementation requirements, utilizing big data technologies and tools.
This is a remote position.
Role: Big Data Engineer/Developer
Experience: 3-6 Years in Big Data
Immediate Joiners: Preferred
Location: Remote( for now)
Job Description:
- Data Engineer required with experience working in Hadoop in technologies including Hive, Hdfs, Sqoop etc.
- Must have - Python, Pyspark, Spark, AWS/Azure, SQL.
- Candidate will perform the role of senior developer where he needs to support existing applications as well as work on new requirements from a design and implementation perspectives.
- GitLab Hadoop Hive Jenkins MAVEN Nexus
- Scala Shell Scripting YARN Oracle Scrum/Agile Spring.
- Sound communication skills - both written and verbal
- Good to have - Knowledge of Teradata, Airflow, GCP
Similar Jobs
Fintech • Financial Services
Designs and delivers scalable data architectures and end-to-end solutions. Builds ETL pipelines using AWS, Spark, Scala, and big data technologies; contributes to technical design and implementation; collaborates with business and engineering teams; supports Agile delivery; promotes engineering best practices; and ensures secure, highly available solutions meet governance and risk standards.
Top Skills:
SparkAWSHadoopHiveHugging FaceLangchainLlamaOracle SqlPl/SqlPythonScalaTeradata
Cloud • eCommerce • HR Tech • Information Technology • Analytics
Manage and support AKS-based enterprise data platform infrastructure, administer Apache NiFi, Azure Event Hub and Functions, implement CI/CD with Azure DevOps, automate infrastructure with Terraform/Bicep/ARM, monitor with Azure Monitor and Log Analytics, troubleshoot production incidents, ensure security, capacity planning, and collaborate with data engineering teams.
Top Skills:
Apache NifiArgocdArm TemplatesAzure Application GatewayAzure Application InsightsAzure Container RegistryAzure Data FactoryAzure DevopsAzure Event HubAzure FirewallAzure FunctionsAzure Kubernetes Service (Aks)Azure MonitorAzure Synapse AnalyticsBashBicepCluster AutoscalerDockerFluxGitGrafanaHpaIstioKqlKubernetesLog AnalyticsMicrosoft Defender For CloudNetwork Security GroupNginxNifi RegistryOpentelemetryPeeringPowershellPrivate EndpointsPrometheusPythonRbacTerraformVirtual NetworkVpaVpnYaml Pipelines
Software
Design, implement, and lead large-scale generative AI and ML systems (LLM fine-tuning, RAG, agents, Graph ML). Build REST APIs, deploy with Docker/Kubernetes, optimize big-data workflows, mentor engineers, and operate ML services on cloud platforms.
Top Skills:
Anomaly DetectionAWSAzureBig DataCi/CdData MiningDockerDocument DatabasesFastapiGenerative AiGCPGraph MlKnowledge BasesKubernetesLlmsNoSQLPythonPyTorchRagRestful ApisSagemakerTensorrtTriton
What you need to know about the Kolkata Tech Scene
When considering the industries shaping India's tech scene, gaming might not immediately come to mind. However, in the last decade, increased internet usage and greater access to mobile devices have catapulted the industry to new heights, with Kolkata-based companies like Virtualinfocom, Red Apple Technologies and Digitoonz, at the forefront, driving the design and animation of new gaming titles for players.
