Senior Data Engineer

Job Title: Senior Data Engineer – Databricks / SparkExperience: 5–7 YearsJob SummaryWe are looking for a Senior Data Engineer with strong hands-on experience in Databricks, Apache Spark, SQL, data warehousing, and lakehouse architecture.

The ideal candidate should be capable of independently owning data pipelines end to end, making sound technical and architectural decisions, and optimizing large-scale data workloads.The role requires a strong engineering mindset with experience in performance optimization, troubleshooting, technical documentation, governance, and mentoring junior engineers.Key ResponsibilitiesDesign, develop, and maintain end-to-end data pipelines using Databricks, Spark, SQL, and related data engineering technologies.Independently own pipeline development from requirements and technical design through implementation, testing, deployment, and production support.Develop complex and optimized SQL queries, data models, transformations, and ETL/ELT processes.Design scalable and maintainable data warehouse and lakehouse solutions based on business and technical requirements.Analyze architecture and design options and make appropriate decisions around scalability, performance, reliability, cost, and maintainability.Troubleshoot and optimize Spark/Databricks workloads at scale, including identifying performance bottlenecks and improving processing efficiency.Establish and follow data engineering standards, coding best practices, development guidelines, and governance processes.Create and maintain technical documentation covering architecture, pipelines, data models, processes, and operational procedures.Conduct code/design reviews and provide technical guidance and mentoring to junior data engineers.Collaborate with architects, developers, analysts, and business stakeholders to deliver reliable and scalable data solutions.Contribute to automation and engineering best practices through CI/CD and Infrastructure as Code (IaC) where applicable.Mandatory Skills5–7 years of hands-on Data Engineering experience.Strong experience with Apache Spark and Databricks.Advanced SQL, including complex queries and performance tuning.Strong understanding of ETL/ELT pipeline development.Good knowledge of data warehousing and lakehouse architecture.Experience with data modelling and scalable data platform design.Proven experience in Spark/Databricks performance optimization and troubleshooting.Ability to independently design and deliver data engineering solutions.Experience providing technical guidance or mentoring to junior engineers.Strong understanding of development standards, technical documentation, and data governance.Good to HaveDatabricks certification.Experience with CI/CD for data engineering projects.Experience with Infrastructure as Code (IaC) such as Terraform.Git-based development and deployment practices.Experience working in Agile engineering environments.Ideal CandidateThe ideal candidate is not only a strong pipeline developer but also someone who can design, optimize, troubleshoot, document, and take ownership of data platforms and solutions end to end.