Website:
naveeratech.com
Job details:
About Naveera:
Naveera Technology LLC is a trusted global engineering partner delivering Data Engineering, Generative AI, Application Development, and IT Infrastructure solutions. With over 15 years of experience in IT services and consulting, we help organizations transform raw data into actionable business value.
With a team of 100+ employees and successful delivery of 5+ global projects, Naveera serves clients across multiple industries and geographies through agile delivery models and proven engineering practices.
We're Hiring: Senior Lead Data Engineer | GCP Cloud
Skills Required: AWS to GCP Migration, Google Pub/Sub, Dataflow (Apache Beam), BigQuery (including streaming inserts), Cloud Composer (Airflow), Cloud Storage (GCS), S3, Glue, Redshift, Python, PySpark, Data Build Tool (DBT), Medallion & Serverless Architecture, Complex SQL Queries
Job Location: Madurai
Mode: Remote (Quarterly travel to the Madurai office is required, with a mandatory 5-working-day stay. Please note that accommodation and travel expenses will not be reimbursed by the company.)
Job Type: Full-Time
Experience: 10 to 15 Years
Shift Timing: 2 PM to 11:30 PM IST (Extended work may be required based on business needs)
Interview Process:
L1 Technical Discussion: Virtual Discussion Via Gmeet.
L2 Technical Discussion: Virtual Discussion Via Gmeet.
CTO Technical Discussion: Virtual Discussion Via Gmeet.
Client Interview: Virtual
HR Discussion: Virtual
Onboarding: Virtual Onboarding
Job Summary
We are seeking a hands-on Senior/ LeadGCP Data Engineer with deep expertise in real-time streaming data architectures to help design, build, and optimize data pipelines in our Google Cloud Platform (GCP) environment. The ideal candidate will have strong architectural vision and be comfortable rolling up their sleeves to build scalable, low-latency streaming data pipelines using Pub/Sub, Dataflow (Apache Beam), and BigQuery.
Key Responsibilities
- Architect and implement end-to-end streaming data solutions on GCP using Pub/Sub, Dataflow, and BigQuery.
- Design real-time ingestion, enrichment, and transformation pipelines for high-volume event data.
- Work closely with stakeholders to understand data requirements and translate them into scalable designs.
- Optimize streaming pipeline performance, latency, and throughput.
- Build and manage orchestration workflows using Cloud Composer (Airflow).
- Drive schema design, partitioning, and clustering strategies in BigQuery for both real-time and batch datasets.
- Define SLAs, monitoring, logging, and alerting for streaming jobs using Cloud Monitoring, Error Reporting, and Stackdriver.
- Experience with the data modeling.
- Ensure robust security, encryption, and access controls across all data layers.
- Collaborate with DevOps for CI/CD automation of data workflows using Terraform, Cloud Build, and Git.
- Document streaming architecture, data lineage, and deployment runbooks.
Required Skills & Experience
- 8+ years of experience in data engineering or architecture.
- 3+ years of hands-on GCP data engineering experience.
- Strong expertise in:
- Google Pub/Sub
- Dataflow (Apache Beam)
- BigQuery (including streaming inserts)
- Cloud Composer (Airflow)
- Cloud Storage (GCS)
- Solid understanding of streaming design patterns, exactly-once delivery, and event-driven architecture.
- Deep knowledge of SQL and NoSQL data modeling.
- Hands-on experience with monitoring and performance tuning of streaming jobs.
- Experience using Terraform or equivalent for infrastructure as code.
- Familiarity with CI/CD pipelines for data workflows.
Click on Apply to know more.
