UST
Website:
ust.com
Job details:
Role Description
Job Summary
We are looking for a skilled Data Engineer to design, build, and maintain scalable data pipelines using modern cloud technologies. The ideal candidate will have hands-on experience with
Google Cloud Dataflow and
Apache Beam, along with a strong understanding of data processing frameworks and cloud-based architectures.
Key Responsibilities
- Design and implement robust batch and real-time data pipelines using Google Cloud Dataflow and Apache Beam (Java/Python).
- Develop scalable data solutions utilizing GCP services such as BigQuery, Google Cloud Pub/Sub, and Google Cloud Storage.
- Monitor pipeline performance and optimize for reliability, scalability, and cost efficiency.
- Troubleshoot production issues and ensure high availability and fault tolerance of data systems.
- Collaborate with data analysts, engineers, and business stakeholders to understand and deliver data requirements.
- Maintain data quality by implementing validation checks and basic governance practices.
- Build and manage CI/CD pipelines for automated deployment and maintenance of data workflows.
Required Skills & Qualifications - Strong experience in Data Engineering and data pipeline development.
- Hands-on expertise with Apache Beam and Google Cloud Dataflow.
- Proficiency in Java or Python for data processing.
- Experience with GCP services, especially BigQuery, Pub/Sub, and Cloud Storage.
- Familiarity with CI/CD tools and practices (e.g., Git, Jenkins, Cloud Build).
- Understanding of data quality, validation, and governance principles.
- Strong problem-solving and performance optimization skills.
Skills
Data Engineering,data pipeline,cloud dataflow,apache beam,cloud platform,cicd,
Click on Apply to know more.