Website:
avinya-technologies.com
Job details:
Job Title: GCP Data Engineer
Experience: 5+ Years
Location: Hyderabad/Pune
Employment Type: Full-Time
About the Role
We are seeking a seasoned GCP Data Engineer with over 5 years of hands-on experience designing, building, and optimizing scalable cloud data platforms. You will play a critical role in architecting robust batch and real-time ETL/ELT pipelines, managing enterprise data warehouses, and ensuring high data reliability across the organization using Google Cloud Platform, Python, and SQL.
Key Responsibilities
- Data Pipeline Architecture: Design, develop, and maintain scalable, fault-tolerant batch and streaming data pipelines using GCP services such as Dataflow, Dataproc, Cloud Composer (Airflow), and Pub/Sub.
- Data Warehousing & Modeling: Architect and optimize large-scale data warehouses in BigQuery, implementing best practices for partitioning, clustering, cost optimization, and dimensional data modeling.
- Programming & Automation: Write clean, modular, and production-grade Python code for data ingestion, transformation, API integrations, and workflow automation.
- Advanced SQL & Transformations: Develop complex SQL queries, stored procedures, and modular transformation layers (using tools like dbt or Dataform) to support business intelligence and analytics.
- Data Governance & Quality: Implement automated data quality frameworks, data profiling, lineage tracking, and strict security/IAM access controls across cloud storage and databases.
- Cross-Functional Collaboration: Partner closely with data scientists, analytics teams, software engineers, and business stakeholders to translate complex business requirements into reliable technical solutions.
- CI/CD & DevOps: Establish and maintain CI/CD pipelines, Infrastructure as Code (Terraform), and monitoring/alerting systems to ensure pipeline observability and minimal downtime.
Required Qualifications & Skills
- Experience: 5+ years of proven experience in data engineering, big data processing, and cloud data platform development.
- GCP Ecosystem: Deep hands-on expertise with core Google Cloud data services, specifically BigQuery, Dataflow, Cloud Composer (Airflow), Cloud Storage (GCS), Pub/Sub, and Cloud SQL.
- Python Proficiency: Strong programming experience in Python for data engineering (e.g., Pandas, PySpark, Apache Beam, and interacting with GCP SDKs/APIs).
- SQL Mastery: Advanced SQL skills with proven experience in complex query writing, performance tuning, and query execution plan optimization.
- Data Formats & Storage: Solid understanding of modern file formats (Parquet, Avro, ORC, JSON) as well as relational (RDBMS) and NoSQL databases.
- Data Modeling: Strong grasp of data warehousing concepts, ETL/ELT best practices, schema design, and data modeling techniques (Star/Snowflake schemas, Data Vault).
- Education: Bachelor’s or Master’s degree in Computer Science, Engineering, Information Systems, or a related quantitative field (or equivalent practical experience).
# Immediate Joiners preferred
Relevant profiles please apply or share your resume at ajita@avinya-technologies.com
Click on Apply to know more.