Birlasoft
Website:
birlasoft.com
Job details:
Area(s) of responsibility
Job Description: Azure Data Engineer
Key Responsibilities
- Data Solution Design & Architecture
- Design end‑to‑end data engineering solutions using Azure Data Factory, Azure Databricks, PySpark, SQL, and other Azure-native services.
- Architect and implement scalable and secure Modern Data Warehouse (MDW) and Lakehouse solutions leveraging Azure Data Lake Storage and Databricks.
- Databricks & PySpark Engineering
- Develop scalable transformation scripts using PySpark on Databricks, applying advanced optimizations like caching, partitioning, and Delta Lake capabilities.
- Implement Delta Lake features—ACID transactions, schema enforcement, schema evolution, and time travel—across the data lifecycle.
- Perform performance tuning, handling bottlenecks related to cluster configuration, shuffle operations, joins, and parallelization.
- Collaborate with platform teams to manage Databricks clusters, jobs, notebooks, and CI/CD integrations.
Click on Apply to know more.