Golden Opportunities
Website:
gojobs.biz
Job details:
Job Description
Job Title: Data Engineer (PySpark + Azure Databricks)
Location: Bangalore, Hyderabad, Chennai & Mumbai
Experience: 6 � 10 years
Job Summary
We are looking for a skilled Data Engineer with strong experience in PySpark and Azure Databricks to design, build, and optimize scalable data pipelines and analytics solutions. The ideal candidate will have hands-on expertise in big data processing, Azure data services, and automation for data-driven decision-making.
Key Responsibilities
- Design, develop, and maintain ETL/ELT data pipelines using Azure Data Factory and Databricks.
- Build and optimize PySpark scripts and Databricks notebooks for large-scale data processing.
- Perform data ingestion, transformation, and aggregation from multiple sources (SQL, APIs, ADLS, Blob, SFTP, etc.).
- Implement Delta Lake architecture for data versioning, ACID transactions, and optimized queries.
- Work with Azure Data Lake Storage (Gen2) for managing and structuring raw, processed, and curated data layers.
- Develop and manage CI/CD pipelines for Databricks using Azure DevOps or GitHub Actions.
- Implement data validation, quality checks, and monitoring mechanisms across data pipelines.
- Collaborate with business and analytics teams to deliver clean, structured data for reporting and machine learning.
- Optimize PySpark jobs using techniques like broadcast joins, Z-Ordering, partitioning, and caching.
- Ensure compliance with data governance, security, and regulatory requirements.
- Troubleshoot performance bottlenecks and improve resource utilization on Databricks clusters.
Skills Required
RoleData Engineer (Pyspark + DataBricks) Contract Hiring
Industry TypeIT/ Computers - Software
Functional Area
Required Education Bachelor Degree
Employment TypeFull Time, Permanent
Key Skills
- DATA ENGINEER
- DATABRICKS
- PY SPARK
Other Information
Job CodeGO/JC/1327/2025
Recruiter NameChristopher
Click on Apply to know more.