Confluent Kafka

Responsibilities

Design, develop, and manage Confluent Kafka solutions hosted on Azure.

Deploy and configure Confluent Cloud on Azure, ensuring optimal performance and scalability.

Integrate Confluent Kafka with Azure-based, Oracle OCI and on-prem enterprise applications.

Implement Confluent Schema Registry to ensure data governance and schema enforcement.

Utilize Kafka Connect for seamless integration with Azure services (e.g., Azure Data Lake, Azure Event Hubs, Cosmos DB).

Develop real-time processing workflows using KSQL and Kafka Streams.

Monitor and optimize Kafka performance using Confluent Control Center and Azure Monitor.

Ensure high availability, fault tolerance, and security using Azure RBAC, Managed Identities, and private networking.

Troubleshoot and resolve data streaming, connectivity, and performance issues in a cloud-native environment.

Provide best practices, documentation, and knowledge transfer to internal teams.

Skills

Must have

3+ years of experience with Confluent Platform (or Apache Kafka in an enterprise setting).

Strong understanding of Kafka topics, partitions, consumer groups, and producer/consumer configurations.

Experience with Confluent Schema Registry for schema validation and governance.

Hands-on experience with Kafka Connect to integrate external systems, including Azure services.

Proficiency in KSQL or Kafka Streams for real-time data transformation.

Understanding of Azure cloud services related to data streaming, such as Azure Event Hubs, Azure Functions, Azure Data Lake, and Cosmos DB.

Experience with Azure networking and security best practices, including private endpoints, RBAC, and Managed Identities.

Familiarity with Azure DevOps pipelines for CI/CD automation.

Strong troubleshooting and performance tuning skills.

Nice to have

Confluent Certified Developer/Admin certification is a plus.

Experience with Confluent Cloud on Azure or self-managed Confluent Kafka clusters in Azure Kubernetes Service (AKS).

Exposure to Terraform for infrastructure as code (IaC).

Mandatory Skills

Confluent Platform (or Apache Kafka)

KSQL or Kafka Streams

Azure Event Hubs, Azure Functions, Azure Data Lake, and Cosmos DB