Willware Technologies
Website:
willwaretech.com
Job details:
Job Title: Lead Kafka Engineer / Confluent Platform Architect (Real-Time Streaming, GCP)
Location: Bengaluru, Chennai, Hyderabad, Pune, Kolkata, Lucknow, Madurai, Nagpur, Vijayawada
Experience: 5 Years
Work Mode: Hybrid
Job Summary We are seeking an experienced Kafka / Confluent Platform Architect to lead the design, development, and deployment of enterprise-grade real-time data streaming solutions. The ideal candidate will architect scalable streaming pipelines, manage Kafka clusters across on-prem and GCP environments, and establish best practices for governance, security, and performance. This role also involves technical leadership of offshore teams and automation using CI/CD and Infrastructure as Code.
Key Responsibilities - Lead the design, development, and deployment of solutions using Apache Kafka and Confluent Platform
- Architect and implement real-time data pipelines and event-driven streaming applications
- Install, configure, manage, and optimize Kafka clusters (on-premises and Google Cloud Platform)
- Ensure high availability, fault tolerance, scalability, and performance tuning of Kafka environments
- Implement and manage Kafka Connect, Schema Registry, and ksqlDB for stream processing
- Define and enforce Kafka best practices for usage, governance, data standards, and security
- Monitor cluster health and troubleshoot issues related to latency, consumer lag, and throughput
- Collaborate with cross-functional teams (onshore/offshore) to deliver robust streaming solutions
- Provide technical leadership, mentor offshore team members, and conduct code/design reviews
- Automate deployments and infrastructure using CI/CD pipelines and Terraform
Required Skills & Experience - 8+ years of experience in data engineering or platform engineering roles
- Strong hands-on expertise with Apache Kafka and Confluent Platform
- Experience building real-time/event-driven data pipelines
- Kafka cluster administration, capacity planning, and performance tuning
- Hands-on with Kafka Connect, Schema Registry, ksqlDB, and stream processing
- Experience with GCP, Kubernetes, and Docker-based deployments
- Proficiency in Infrastructure as Code (Terraform) and CI/CD automation
- Strong understanding of Kafka security (SSL, ACLs, RBAC)
- Experience working in distributed onshore/offshore team models
- Excellent troubleshooting and problem-solving skills
Click on Apply to know more.