Senior Data Engineer
Confidential
- Location
- Pune District, Maharashtra, India
- Job type
- Full-time
Required skills
- Python
- BigQuery
- clustering
- data engineer
- Dataproc
- ETL
- GCP
- GCS
- Google Cloud
- Spark
- SQL
- T-SQL
About the role
Website:
Job details:
Summary – Senior Data Engineer Role
- Purpose of the Role:
- Lead the migration of existing on-premise SQL Server ODS/EDW systems to Google Cloud Platform (BigQuery) while modernizing legacy ETL processes into scalable cloud-native data pipelines.
- Key Responsibilities:
- Design and execute the migration strategy from SQL Server/SSIS to BigQuery with minimal downtime.
- Modernize legacy SSIS pipelines using Cloud DataProc (Spark), Cloud Functions, and BigQuery features.
- Build scalable data orchestration and storage solutions using Google Cloud Storage (GCS) and event-driven triggers.
- Optimize SQL queries, partitioning, and clustering in BigQuery for performance and cost efficiency.
- Provide technical leadership and mentorship to junior developers and offshore teams, including code reviews and best practices for CI/CD and governance.
- Technical Skills Required:
- Expert SQL (T-SQL & GoogleSQL).
- Hands-on experience with GCP tools: BigQuery, Cloud DataProc, Cloud Functions, GCS.
- Strong background in SQL Server and SSIS.
- Python for cloud-based processing.
- Experience designing ETL/ELT architectures in cloud environments.
- Understanding of retail datasets such as Sales, Inventory, Purchase Orders, Demand/Supply, Product, and Customer data.
Click on Apply to know more.
This page is fully interactive when JavaScript is enabled. Please enable JavaScript to apply or browse related roles.