Website:
mycareernet.in
Job details:
Company:IT Services Organization
Key Skills: Pyspark, Azure ADF, Azure Databricks, SQL, Python, Azure Data lake, Azure Devops, NoSql, Generative AI, Azure Cloud
Roles and Responsibilities:
- Design, develop, and maintain data pipelines using Azure Data Factory (ADF) and Azure Data Lake Storage (ADLS).
- Implement data processing solutions using Pyspark and Azure Databricks.
- Collaborate with DevOps teams to establish CI/CD practices for data workflows.
- Write and optimize SQL queries for data extraction and transformation.
- Ensure data quality and integrity throughout the data lifecycle.
- Explore and integrate NoSQL databases as needed.
- Stay updated with the latest trends in Generative AI and its applications in data engineering.
Skills Required:
- Expertise in data engineering on Azure platforms (ADF, ADLS, Databricks)
- Strong programming skills in Python and Pyspark
- Experience with Azure DevOps, including CI/CD practices for data workflows
- Proficiency in SQL for data extraction, transformation, and analysis
- Knowledge of NoSQL databases and integration into data pipelines
- Ability to ensure data quality and integrity throughout the data lifecycle
- Familiarity with cloud-based data storage and processing solutions
- Awareness of Generative AI applications in data engineering
- Strong problem-solving, analytical, and collaborative skills
Education: Bachelor's degree in Computer Science, Information Technology, or a related field
Click on Apply to know more.