NSKH Tech
Website:
nskhtech.com
Company:
https://www.linkedin.com/company/nskh
Seniority: Mid-Senior level
Industries: Technology, Information and Internet
Job details:
Job Title: Confluent Flink Engineer
Experience: 6+ Years
Location: Remote
Shift Timings: 2:00 PM – 11:00 PM IST
Role Overview
We are seeking an experienced
Confluent Flink Engineer with strong expertise in real-time data streaming, event-driven architectures, and large-scale distributed systems. The ideal candidate should have hands-on experience with
Apache Flink, Confluent Kafka, Kafka Streams, and streaming ETL pipelines, along with a strong understanding of cloud-native data processing solutions.
The role involves designing, developing, and optimizing real-time streaming applications that process high-volume data efficiently while ensuring scalability, fault tolerance, and performance.
Key ResponsibilitiesStream Processing & Data Engineering
- Design, develop, and maintain real-time data processing applications using Apache Flink.
- Build and manage large-scale streaming ETL pipelines.
- Develop event-driven architectures using Confluent Kafka and Kafka Streams.
- Implement real-time data transformation, aggregation, enrichment, and analytics workflows.
Kafka & Confluent Platform
- Configure and manage Confluent Kafka environments.
- Work with Kafka Connect, Schema Registry, and other Confluent Platform components.
- Design efficient topic structures, partitioning strategies, and consumer group architectures.
- Ensure reliable event processing and message delivery.
Performance & Scalability
- Optimize Flink jobs for performance, throughput, and resource utilization.
- Implement checkpointing, state management, and fault-tolerant processing strategies.
- Monitor and troubleshoot streaming applications and data pipelines.
- Ensure scalability and high availability of streaming platforms.
Cloud & DevOps
- Deploy and manage streaming workloads on AWS, Azure, or GCP.
- Participate in CI/CD pipeline implementation and automation.
- Collaborate with DevOps teams to improve deployment, monitoring, and operational practices.
Collaboration
- Work closely with Data Engineers, Architects, Product Teams, and Business Stakeholders.
- Participate in architecture discussions and technical design reviews.
- Contribute to best practices, documentation, and knowledge sharing.
Required Skills & Experience - 6+ years of software engineering or data engineering experience.
- Strong hands-on experience with Apache Flink.
- Extensive experience with Confluent Kafka and Kafka Streams.
- Strong understanding of real-time streaming and event-driven architectures.
- Experience with distributed systems and large-scale data processing.
- Proficiency in Java, Scala, or Python.
- Experience building and managing streaming ETL pipelines.
- Strong knowledge of:
- Kafka Connect
- Schema Registry
- Confluent Platform Components
- Strong understanding of:
- Data Partitioning
- State Management
- Checkpointing
- Fault Tolerance
- Experience with AWS, Azure, or GCP cloud platforms.
- Familiarity with CI/CD pipelines and DevOps practices.
Preferred Skills
- Experience with Kubernetes and containerized deployments.
- Knowledge of data lake and modern data platform architectures.
- Exposure to monitoring tools such as Prometheus, Grafana, or Datadog.
- Experience with Spark Streaming or other stream processing technologies.
Key Competencies
- Real-Time Data Streaming
- Apache Flink Development
- Confluent Kafka & Kafka Streams
- Distributed Systems
- Event-Driven Architecture
- Streaming ETL Pipelines
- Cloud Data Engineering
- Performance Optimization
- CI/CD & DevOps
Skills: pipelines,etl,data,cloud,devops,processing,kafka,flink
Click on Apply to know more.
