UST
Website:
ust.com
Job details:
Role Description
Job Role: Data Architect
Job Location: PAN India
Experience Required: 6+ years
Notice Period: Immediate Joiners Only
Job Summary
We are seeking a highly skilled Data Engineer with 6+ years of experience in building scalable big data and cloud-based data engineering solutions. The ideal candidate will have strong expertise in Databricks, Apache Spark, ETL/ELT development, and modern data lake/lakehouse architectures.
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT pipelines using Databricks and distributed data processing frameworks.
- Build and optimize big data solutions using Apache Spark with Python, Scala, or SQL.
- Develop and manage data lake/lakehouse architectures on cloud platforms such as AWS, Azure, or GCP.
- Implement batch and real-time data ingestion pipelines from structured and unstructured data sources.
- Optimize Databricks performance, cluster management, and cost efficiency.
- Create reusable notebooks, workflows, and automated jobs using Databricks Workflows and CI/CD practices.
- Work with Delta Lake, data modeling, partitioning, and schema evolution for reliable and efficient processing.
- Collaborate with data analysts, data scientists, and business teams to deliver enterprise-grade data solutions.
- Ensure data governance, security, monitoring, and compliance across data platforms.
- Troubleshoot production issues and automate operational data engineering tasks.
Required Skills
- 6+ years of experience in Data Engineering and Big Data technologies.
- Strong hands-on experience with Databricks and Apache Spark.
- Proficiency in Python, Scala, or SQL for data processing.
- Experience building ETL/ELT pipelines and distributed data processing solutions.
- Strong understanding of Delta Lake, Lakehouse Architecture, and Data Modeling.
- Experience with cloud platforms: AWS, Azure, or GCP.
- Knowledge of batch and real-time data ingestion frameworks.
- Familiarity with CI/CD, workflow orchestration, and performance tuning.
- Strong problem-solving and collaboration skills.
Preferred Skills
- Experience with streaming technologies such as Kafka or Spark Streaming.
- Exposure to DevOps and Infrastructure as Code practices.
- Knowledge of data governance and security best practices.
Skills
Data Engineering, Data Processing, Apache Spark, Databricks
Click on Apply to know more.