Technogen India Pvt. Ltd.
Website:
technogenindia.com
Job details:
Data Engineer | PySpark | Apache Iceberg | AWS EMR | Airflow | Kafka
About the Role
We are looking for an experienced Data Engineer to join our growing data engineering team. The ideal candidate should have strong expertise in building and supporting scalable data pipelines using modern Big Data and Cloud technologies.
This is an excellent opportunity to work on cutting-edge data platforms, optimize large-scale data processing, and contribute to enterprise-grade analytics solutions.
Key Responsibilities
- Design, develop, and optimize ETL/data processing pipelines using PySpark and Python.
- Build, maintain, and orchestrate workflows using Apache Airflow.
- Manage Apache Iceberg tables including schema evolution, partitioning, bucketing, and time travel.
- Execute and optimize Spark workloads on AWS EMR.
- Develop and support Kafka-based CDC (Change Data Capture) ingestion pipelines.
- Write optimized SQL queries and implement transformation logic using Medallion Architecture (Bronze, Silver, Gold).
- Troubleshoot production issues and provide L3 Support for data platforms.
- Collaborate with cross-functional teams to improve data quality, scalability, and performance.
- Follow Git version control and CI/CD best practices throughout the development lifecycle.
Required Skills
- 6–10 years of experience as a Data Engineer
- Strong hands-on experience with PySpark and Python
- Expertise in Apache Iceberg
- Experience with Apache Airflow
- Hands-on experience with AWS EMR
- Strong SQL skills
- Experience with Kafka and CDC concepts
- Knowledge of Git and CI/CD
- Experience in production support and troubleshooting
Click on Apply to know more.