Senior Data Engineer

Senior Data Engineer – Spark / Scala

Location: Bangalore, IndiaIndustry: Retail & E-commerceAbout the RoleWe are looking for an experienced Senior Data Engineer to join our data engineering team supporting a large-scale US retail environment.

The ideal candidate will have strong hands-on experience building and optimizing data platforms, developing high-volume data pipelines, and working with distributed data processing technologies.This is a hands-on role for someone who can ramp up quickly, troubleshoot complex production issues, and independently drive technical deliverables from development through deployment.Key ResponsibilitiesDesign, develop, and optimize large-scale data pipelines using Apache Spark and Scala.Build and maintain high-volume batch and streaming data pipelines.Develop data processing solutions using Spark, Hive, Kafka, and GCP services.Work with BigQuery, GCS, Dataproc, and Pub/Sub as part of cloud-based data platforms.Perform Spark job optimization, performance tuning, and troubleshooting.Design and maintain scalable ETL/ELT workflows and data pipelines.Troubleshoot complex data, application, performance, and production issues.Support cloud migration and modernization of legacy/on-premise data platforms.Develop reusable, scalable, maintainable, and production-ready data solutions.Implement and maintain CI/CD pipelines using Git, Jenkins, and related DevOps practices.Participate in code reviews, testing, production deployments, and operational support.Collaborate with engineering, analytics, product, and platform teams to deliver end-to-end data solutions.Analyze data quality issues and implement appropriate validation and reconciliation processes.Quickly understand existing systems and take ownership of assigned technical deliverables.Required Qualifications7+ years of experience in Data Engineering, Big Data, or related engineering roles.Strong hands-on experience with Apache Spark and Scala.Strong experience with Hive and distributed data processing.Hands-on experience with Kafka and streaming/event-driven architectures.Strong programming experience in Scala and/or Python.Proven experience designing and troubleshooting large-scale ETL/data pipelines.Strong understanding of distributed systems, data modeling, data quality, and performance optimization.Experience with at least one major cloud platform: GCP, AWS, or Azure.Experience with Git and CI/CD practices.Experience with workflow/orchestration tools such as Oozie or equivalent.Strong production troubleshooting and problem-solving skills.Ability to work independently in a fast-paced engineering environment.Preferred QualificationsStrong experience with Google Cloud Platform (GCP).Experience with BigQuery, GCS, Dataproc, and Pub/Sub.Experience migrating on-premise Spark/Hive workloads to GCP.Experience with large-scale data migration, validation, and reconciliation.Knowledge of data quality, monitoring, and observability frameworks.Experience supporting complex retail, e-commerce, or high-volume transactional data environments.Experience working with multiple upstream and downstream data integrations.Experience with modern data engineering and cloud modernization initiatives.Apply now or reach out with your resume to learn more about the opportunity at karobi.h@andortech.com#Hiring #DataEngineer #SeniorDataEngineer #BigData #ApacheSpark #Scala #Kafka #Hive #GCP #BigQuery #Dataproc #DataEngineering #CloudComputing #ETL #Python #Jenkins #CICD #RetailTech #TechJobs #USJobs