Talentmatics
Website:
talent-matics.com
Job details:
Our client is looking for an experienced Azure Databricks Engineer / Data Engineer to join their growing data and analytics team. The ideal candidate will be responsible for designing, developing, and maintaining scalable data pipelines and ETL processes using Azure and Databricks technologies.
Key Responsibilities:
- Design, develop, and maintain scalable ETL/data pipelines using Azure Databricks and Azure Data Factory (ADF).
- Develop data processing and transformation workflows using SQL, Python, Spark, and Databricks notebooks.
- Integrate and orchestrate data from multiple data sources while ensuring data quality and consistency.
- Build reusable frameworks and rules for data quality validation and data transformations.
- Monitor, optimize, and troubleshoot data pipelines for performance, scalability, and reliability.
- Work closely with business stakeholders, application SMEs, data analysts, and data scientists to gather and understand business requirements.
- Analyze source systems and define technical specifications for BI and Data Warehouse solutions.
- Create conceptual and logical data models aligned with business processes and scalable architecture practices.
- Ensure compliance with data governance, security, quality, and regulatory standards.
- Maintain proper project documentation and follow development best practices.
Required Skills & Expertise:
- Strong hands-on experience with Azure Databricks and Azure Data Factory (ADF).
- Expertise in SQL, Python, Spark, and Scala.
- Strong experience in building optimized and reusable ETL/data pipeline frameworks.
- Good understanding of Delta Lake, Spark SQL performance tuning, and Databricks cluster configurations.
- Experience with data integration, data migration, data validation, and data warehousing concepts.
- Knowledge of self-hosted integration runtimes, SSIS, and Azure integration components.
- Familiarity with BI/reporting tools such as Power BI and Qlik Sense.
- Strong analytical, troubleshooting, and problem-solving skills.
- Excellent communication and stakeholder management abilities.
- Bachelor’s degree in Computer Science, Information Technology, or related field.
Preferred Experience:
- Experience working in enterprise-scale Azure data environments.
- Understanding of modern data engineering and big data best practices.
- Ability to work in a fast-paced, collaborative environment with a proactive approach.
Click on Apply to know more.