Talentgigs
Website:
talentgigs.in
Job details:
Description
Strong expertise in Azure Data Services: o Azure Data Lake, Azure Databricks, Azure Synapse, Azure Data Factory. • Proficiency in Python and PySpark for large-scale data processing. • Solid understanding of data lakehouse architectures, Delta Lake, and parquet formats. • Experience with ETL/ELT design, data modeling, and pipeline orchestration. • Familiarity with SQL (T-SQL, Spark SQL) for querying and transformations. • Knowledge of CI/CD practices and source control (Azure DevOps, Git, GitHub). • Strong problem-solving skills with ability to debug complex data issues.
Key Responsibilities:
- Develop and optimize ETL/ELT data pipelines using Python,PySpark within Azure Databricks.
- Design and implement data solutions leveraging Azure Data Lake Storage and Databricks.
- Ensure data quality, governance, and security, utilizing Databricks Unity Catalog (UC).
- Write and optimize complex SQL queries for data manipulation and analysis.
- Integrate DevSecOps practices into data engineering workflows for automation and security.
Collaborate with cross-functional teams to meet data requirements
Required Skills
Required Skills Key Responsibilities: • Develop and optimize ETL/ELT data pipelines using Python,PySpark within Azure Databricks. • Design and implement data solutions leveraging Azure Data Lake Storage and Databricks. • Ensure data quality, governance, and security, utilizing Databricks Unity Catalog (UC). • Write and optimize complex SQL queries for data manipulation and analysis. • Integrate DevSecOps practices into data engineering workflows for automation and security. • Collaborate with cross-functional teams to meet data requirements
KRAs
KRAs Key Responsibilities: • Develop and optimize ETL/ELT data pipelines using Python,PySpark within Azure Databricks. • Design and implement data solutions leveraging Azure Data Lake Storage and Databricks. • Ensure data quality, governance, and security, utilizing Databricks Unity Catalog (UC). • Write and optimize complex SQL queries for data manipulation and analysis. • Integrate DevSecOps practices into data engineering workflows for
Click on Apply to know more.