Senior Event Streaming Engineer (Kafka / Azure Event Hubs)

Position: Senior Event Streaming Engineer (Kafka / Azure Event Hubs)Location: RemoteContract: Long termRole DescriptionWe are seeking a hands-on Senior Engineer / Lead to design and implement an event-driven integration capability using Azure Event Hubs and Kafka-compatible interfaces.This role would own the integration design, Event Hubs configuration, security and access controls, event ingestion, and downstream consumer enablement. The role would collaborate closely with Cloud Engineering, Information Security, and application teams, with Azure infrastructure provisioning and network configuration handled by the Cloud Engineering team.ResponsibilitiesRequirements and DesignAssess upstream event producer capabilities, Kafka endpoints, topics, schemas, sample payloads, throughput, latency, and ordering requirements.Design the integration approach, including direct publishing versus a Kafka bridge, partitioning strategy, availability, and disaster recovery.Event Hubs Configuration and SecurityDefine and configure event hubs, partitions, consumer groups, retention, and throughput settings; coordinate infrastructure provisioning with Cloud Engineering.Implement appropriate authentication, RBAC, least-privilege access, encryption controls, and secure credential management.Event Ingestion and Consumer EnablementImplement event ingestion, topic mapping, serialization/deserialization, schema compatibility, and event metadata handling.Enable downstream consumers using Kafka protocol or Azure-native SDKs, including independent consumer groups and checkpoint management.Reliability, Testing, and OperationsImplement retry, recovery, duplicate handling, and event replay strategies; validate end-to-end event delivery.Establish monitoring, alerting, operational runbooks, and technical documentation for the foundational implementation.QualificationsA. Mandatory QualificationsHands-on experience designing and implementing solutions using Azure Event Hubs and Apache Kafka.Strong understanding of Kafka producers/consumers, topics, partitions, partition keys, consumer groups, offsets, and delivery semantics.Experience with event-driven architecture, partitioning strategies, serialization/deserialization, and schema compatibility.Practical knowledge of Azure security, including Microsoft Entra ID, RBAC, authentication, authorization, encryption, and secure credential management.Experience implementing reliable event processing, including retries, duplicate handling, recovery, and replay.Ability to independently design, implement, troubleshoot, and test end-to-end integration solutions.Preferred QualificationsProgramming proficiency in C#/.NET for Kafka integration or bridge development.Experience with JSON, Avro, schema registries, and event schema versioning.Familiarity with Azure Monitor, Log Analytics, Terraform/Bicep, and CI/CD automation.Knowledge of high availability, disaster recovery, and production support; experience in financial services or other regulated environments.Relevant Azure or Kafka certifications, particularly:Confluent Certified Developer for Apache Kafka (CCDAK)Confluent Cloud Certified Operator (CCAC)