AWS Senior Data Engineer

AWS Senior Data Engineer (Kafka & PySpark)Experience: 10-15 Years

Location: Chennai, Bengaluru, HyderabadWe are seeking an experienced AWS Data Engineer with strong expertise in Kafka, PySpark, Spark SQL, Python, and AWS to build and optimize large-scale batch and real-time data platforms.Key SkillsPySpark, Spark SQL, Python, SQLApache Kafka (Kafka Connect, Schema Registry, Streaming)AWS: S3, Glue, EMR, Lambda, IAM, Redshift, Step Functions, CloudWatchETL/ELT & Real-Time Data PipelinesSpark Structured StreamingData Modeling, Data Lakes & Lakehouse ArchitectureCI/CD, Git, Jenkins/GitHub ActionsData Quality, Governance & SecurityResponsibilitiesDesign and develop scalable batch and streaming data pipelines on AWS.Build high-performance PySpark and Spark SQL applications.Implement Kafka-based event-driven and real-time streaming solutions.Integrate and optimize AWS data services for reliability and cost efficiency.Implement data quality, governance, monitoring, and security best practices.Build CI/CD pipelines and support production deployments.Troubleshoot performance bottlenecks and production issues.Good to HaveAmazon MSK, KinesisDatabricks, Delta Lake, Unity CatalogAirflow / MWAASnowflake, DBTTerraformPrimary Skills: AWS Data Engineering, Kafka, PySpark, Spark SQL, Python, Streaming Data Engineering.