Skillset- Strong expertise in GCP (BigQuery, Composer, Dataproc, Dataflow, Pub/Sub)
- Hands-on experience with Databricks for large-scale data processing
- Strong programming skills in SQL and PySpark
- Experience with Oracle and SQL Server databases
- Strong understanding of data modelling (dimensional, medallion, UDM)
- Experience in batch and streaming pipeline development
- Knowledge of data ingestion, CDC, and orchestration frameworks
- Familiarity with data governance, quality, and lineage
- Exposure to CI/CD and DevOps practices
- Strong leadership and stakeholder management skills
Detailed Responsibilities- Design and build scalable batch and streaming data pipelines
- Develop data ingestion and transformation frameworks
- Implement CDC and incremental data loading strategies
- Work on GCP platforms including BigQuery, Dataflow, Dataproc, and Pub/Sub
- Build and manage workflows using Cloud Composer (Airflow)
- Implement metadata-driven and reusable pipeline frameworks
- Ensure data quality, validation, and monitoring
- Drive migration from Oracle/SQL Server to cloud platforms
- Convert legacy SQL and ETL logic to BigQuery/PySpark
- Collaborate with analytics and reporting teams
- Lead and mentor data engineering teams
- Optimise pipelines for performance and cost
- Troubleshoot and resolve pipeline/data issues
- Support advanced analytics and AI/ML use cases
Location: Chennai / Bengaluru / Hyderabad
Experience: 8+ Years |