Apach Kafka Administrator

Job SummaryWe are seeking an experienced Senior Kafka Engineer to design, develop, implement, and maintain enterprise-grade event streaming platforms using Apache Kafka.The ideal candidate will have strong hands-on expertise in Kafka architecture, real-time data streaming, distributed systems, and cloud-native technologies. This role will be instrumental in building scalable, highly available, and secure event-driven solutions while establishing engineering standards and best practices across the organization.The candidate will also provide technical leadership, drive architectural decisions, optimize platform performance, and collaborate with cross-functional teams to support business-critical real-time data integration initiatives.Key ResponsibilitiesDesign, develop, and implement scalable, fault-tolerant event-driven architectures using Apache Kafka.Build and manage real-time data streaming pipelines and distributed streaming applications.Administer, configure, monitor, and optimize Kafka clusters for performance, availability, and security.Implement and manage Kafka ecosystem components, including Kafka Connect, Kafka Streams, Schema Registry, and Confluent Platform.Establish standards for topic management, partitioning, replication, retention policies, and data governance.Perform capacity planning, performance tuning, cluster upgrades, and infrastructure optimization.Troubleshoot complex production issues and conduct root cause analysis.Implement automation using Infrastructure as Code and CI/CD pipelines.Collaborate with application development, DevOps, cloud, infrastructure, and data engineering teams.Drive architectural reviews, technical discussions, and engineering best practices.Mentor junior engineers and provide technical guidance to project teams.Ensure compliance with enterprise security, operational, and data governance standards.Required Technical Skills1. Apache Kafka – Primary SkillMinimum 5 years of hands-on experience with Apache Kafka.Strong understanding of Kafka architecture, brokers, topics, partitions, replication, and consumer groups.Expertise in producer and consumer APIs, offset management, and message delivery semantics.Hands-on experience with Kafka Connect, Kafka Streams, and Schema Registry.Knowledge of Confluent Platform and ksqlDB.Experience in Kafka cluster administration, monitoring, troubleshooting, and performance tuning.2. Programming & DevelopmentStrong programming experience in Java and Spring Boot.Experience developing event-driven microservices and distributed applications.Python knowledge is an added advantage.3. Cloud & DevOpsExperience with at least one major cloud platform: AWS, Azure, or GCP.Hands-on experience with Docker and Kubernetes/OpenShift.Experience with Terraform or other Infrastructure as Code tools.Familiarity with CI/CD tools such as Jenkins, GitHub Actions, or Azure DevOps.4. Monitoring & ObservabilityExperience with Prometheus and Grafana.Knowledge of ELK Stack and Splunk.Ability to monitor Kafka performance, troubleshoot bottlenecks, and ensure platform reliability.Leadership & Technical ResponsibilitiesLead Kafka architecture and solution design discussions.Define platform engineering standards, governance, and best practices.Conduct code reviews and ensure adherence to engineering standards.Guide development teams in implementing event-driven solutions.Engage with business and technology stakeholders to translate requirements into scalable streaming architectures.Qualifications & ExperienceBachelor's or Master's degree in Computer Science, Information Technology, or a related discipline.8–12 years of overall IT experience.Minimum 5 years of hands-on Apache Kafka experience.Proven experience supporting mission-critical, enterprise-scale applications.Strong experience in distributed systems, event-driven architecture, and real-time data processing.Preferred CertificationsConfluent Certified Developer for Apache Kafka.Confluent Certified Administrator for Apache Kafka.AWS, Azure, or GCP certifications.Certified Kubernetes Administrator (CKA) or Certified Kubernetes Application Developer (CKAD)