Happiest Minds Technologies
Website:
happiestminds.com
Job details:
Job Description??
Job Title:?Data Engineer
Location:?Bengaluru (In Office)?
Azure Databricks, Azure Data lake, SQL Server
- AB InBev?is?seeking a skilled Azure Data Engineer with a minimum of 4 years of experience to join our dynamic team. The ideal candidate?should?have a strong background in Azure Data Factory (ADF), Databricks, and Delta tables, along with?proficiency?in?PySpark?and SQL. Experience with Feature Store is considered a significant advantage.???
- Key tasks & accountabilities??
- Responsible for Creation of ETL Pipelines.?
- Responsible for designing and building new Data Models and?optimizing?existing data models.?
- Exploit elasticity and agility of the cloud.?
- Ownership of physical Data models of serving layer?
- Responsible and accountable for pipelines reliability and availability.?
- Responsible of Data analysis for troubleshooting and exploration?
- Understand the meaning,?domain?and the context of the data.?
- Professional & Technical skills?
- Essential:??
- Azure Technology Stack: SQL DW/Synapse- Azure Data Factory - Azure Databricks - Azure Data Lake Storage.?
- Ability to write complex SQL scripts and stored procedures.?
- Demonstrated experience in design and delivering data platforms for Business Intelligence or Data Warehouse, including data ingestion, ETL and data integration.?
- Strong skills in handling and analysing complex, high volume data with excellent attention in details.?
- Familiar with Agile?methodology?and Agile working environment.?
- Ability to work along with team leads,?BAs, data architects, other.?
- Develop, design, and implement data pipelines using Azure Data Factory for efficient data ingestion, transformation, and processing.?
- Utilize Databricks to build scalable and optimized data processing workflows and perform advanced analytics tasks.?
- Design and manage Delta tables for handling large-scale?structured and semi-structured data efficiently.?
- Collaborate with cross-functional teams to understand data requirements and translate them into technical solutions.?
- Optimize?and fine-tune?PySpark?scripts for performance improvement and scalability.?
- Create and?maintain?SQL queries for data extraction, manipulation, and analysis.?
- Leverage Feature Store for efficient management and sharing of features across machine learning pipelines (added advantage).?
- Monitor and troubleshoot data pipeline issues to ensure data integrity and reliability.?
- Stay updated with the latest Azure data engineering technologies and best practices.?
- Leverage Git, manage Release Cycles.?
- Qualifications, Experience, Skills?
- Bachelor's degree in Computer Science, Information Technology, or related field.?
- Minimum of 4 years of professional experience as a Data Engineer, with a focus on Azure technologies.?
- Proficiency?in Azure Data Factory for building and orchestrating data pipelines.?
- Strong experience with Databricks for data processing, analytics, and machine learning tasks.?
- Solid understanding of Delta tables and their usage for managing large-scale data sets.?
- Expertise?in?PySpark?for data manipulation, transformation, and analysis.?
- Proficient in SQL with hands-on experience in writing complex queries.?
- Experience with?Feature?Store is a significant advantage.?
- Strong analytical and problem-solving skills with attention to detail.?
- Excellent communication and collaboration skills to work effectively in a team environment.?
- Azure certification(s) such as Azure Data Engineer Associate is a plus.?
- Desirable:?
- Experience in working on AI and ML related projects to prepare the necessary data and data models.
Click on Apply to know more.