Kafka Administrator

We are seeking an experienced Confluent Kafka Administrator & Streaming Engineer to join our Core Infrastructure & Engineering team. In this role, you will own the end-to-end management, scaling, security, and optimization of our enterprise Confluent Kafka streaming platform. You will play a pivotal role in maintaining high availability (HA), disaster recovery (DR), and automation across our real-time data pipelines and cloud environments. Key Responsibilities1. Cluster Administration & OperationsManage, configure, and maintain production enterprise Confluent Kafka clusters, brokers, partitions, and topics. Manage partition rebalancing, topic configuration, consumer group lag monitoring, and data retention policies. Execute platform patching, zero-downtime upgrades, and lifecycle management for Confluent Platform components. Perform capacity planning, node scaling, and performance tuning for high-throughput, low-latency streaming. 2. Security, Resilience & High AvailabilityImplement robust Kafka security controls, including SSL/TLS encryption, SASL authentication, and fine-grained ACLs / RBAC. Design and maintain fault-tolerant architecture with active High Availability (HA) and Multi-Region / Disaster Recovery (DR) strategies. Monitor broker health, cluster metrics, and system throughput using enterprise observability tools. 3. Ecosystem & Stream ProcessingDeploy and manage Kafka Connect (source/sink connectors) for heterogeneous data integrations. Oversee schema evolution and governance using Confluent Schema Registry. Administer stream processing platforms including ksqlDB / KSQL and Apache Flink environments. 4. Automation, DevOps & EngineeringAutomate infrastructure provisioning, configuration management, and deployment pipelines using Ansible, Python, and Bash. Integrate Kafka operations into automated CI/CD pipelines and infrastructure-as-code practices. Author comprehensive operational runbooks, architecture diagrams, and incident resolution guides. Collaborate closely with DevOps, Data Engineering, and cloud teams to support real-time data streaming architectures. Required Skills & QualificationsKafka Core Expertise: Deep experience administering multi-node Apache Kafka or Confluent Platform clusters. Confluent Ecosystem: Hands-on experience with Confluent Connectors, Schema Registry, ksqlDB, and Apache Flink. Security & Infrastructure: Demonstrated background implementing SSL/TLS, SASL, and ACLs on Linux-based environments. Automation & Scripting: Strong proficiency in Python, Bash, and configuration management via Ansible. Cloud & DevOps: Functional experience with major cloud providers (Azure preferred, AWS highly regarded) and CI/CD pipelines. Data Engineering: Sound understanding of distributed systems, real-time message streaming patterns, and pipeline architecture. Troubleshooting: Strong analytical skills for diagnosing producer/consumer failures, broker degradation, and network bottlenecks