Talent Sutra
Website:
talentsutra.co
Job details:
About the Role
AGIVANT is looking for a Senior Software Engineer to design and develop high-performance distributed data ingestion systems. In this role, you will work on parallel processing pipelines that integrate with cloud storage platforms, relational databases, and streaming systems. This is a core product development role where you will apply best practices in distributed system design to build scalable, reliable, and high-performance data platforms.
Key Responsibilities
- Design and implement distributed data ingestion pipelines using Java, C++, or Golang.
- Build parallel data processing systems capable of handling large-scale workloads.
- Develop ingestion pipelines from sources such as Amazon S3, Azure Cloud Storage, Google Cloud Storage, Snowflake, BigQuery, PostgreSQL, Kafka, files, and Iceberg data lakehouse.
- Implement high availability (HA) ingestion systems with cross-region replication.
- Build monitoring, logging, and error reporting mechanisms for ingestion pipelines.
- Integrate ingestion systems with Spark connectors and manage third-party tools such as Kafka and Kafka Connect. • Collaborate with cross-functional teams to ensure scalability, reliability, and performance of distributed systems.
- Follow Agile development practices and contribute to CI/CD pipelines.
Required Skills & Qualifications
- 6–9 years of experience in software development.
- Strong programming expertise in Java, C++, or Golang.
- Hands-on experience with Kafka, Zookeeper, Spark, or stream processing frameworks.
- Expertise in Kafka Connect, Kafka Streams, Kafka security, and customization.
- Experience building event-driven architectures.
- Familiarity with Agile development methodologies.
- Experience working with CI/CD pipelines.
Nice to Have
- Experience with gRPC protocol and multi-threading.
- Exposure to Zookeeper, ETCD, or Consul.
- Understanding of distributed consensus algorithms such as Paxos or Raft.
- Experience with Docker and Kubernetes for containerized deployments.
Click on Apply to know more.