Innova ESI
Website:
innovaesi.com
Job details:
- Design and build scalable ETL/ELT data pipelines using Python and GCP services.
- Develop and optimize batch and real-time data processing workflows.
- Work with structured and unstructured data from multiple sources.
- Implement data ingestion, transformation, and orchestration solutions.
- Build cloud-native data architectures on GCP.
- Collaborate with Data Analysts, Data Scientists, and DevOps teams.
- Ensure data quality, governance, security, and performance optimization.
- Monitor and troubleshoot production data pipelines.
Required Skills
Programming & Data Engineering
- Strong proficiency in:
- Python
- SQL
- PySpark
- Data Structures & OOP concepts
- Experience in:
- ETL/ELT pipeline development
- Data modeling
- Data warehousing concepts
GCP Services
Hands-on experience with:
- BigQuery
- Cloud Storage
- Pub/Sub
- Dataflow
- Composer (Airflow)
- Dataproc
- Cloud Functions
- Cloud Run
- IAM & VPC
Big Data & Tools
- Apache Spark / PySpark
- Apache Airflow
- Kafka (preferred)
- Docker & Kubernetes
- Terraform
- Git/GitHub
Click on Apply to know more.