iDwteam LLC
Website:
idwteam.com
Job details:
Experience: 4–6 Years
Role: Microsoft Fabric Data Engineer
Employment Type: Full-time
Work Location : Hyderabad
Work Model : Work from Office
Job Summary
We are looking for an experienced Microsoft Fabric Data Engineer with 4–6 years of experience in data engineering, data integration, and analytics solutions. The ideal candidate should have strong expertise in SQL, ETL/ELT development, data warehousing, and hands-on experience with Microsoft Fabric to build scalable and reliable data solutions.
Roles and Responsibilities
● Design, develop, and maintain scalable data pipelines using Microsoft Fabric Data Factory.
● Implement Lakehouse and Warehouse solutions using OneLake and Microsoft Fabric architecture.
● Develop ETL/ELT workflows to ingest, transform, and process data from multiple structured and unstructured sources.
● Work with Fabric Notebooks using Python, PySpark, and Spark SQL for large-scale data processing.
● Develop and optimize complex SQL queries, stored procedures, and data transformation logic.
● Build and manage Dataflows Gen2, pipelines, Lakehouse tables, and Warehouse objects.
● Implement incremental data loads, error handling, logging, and pipeline monitoring.
● Optimize Spark jobs, SQL queries, and data pipelines for performance and cost efficiency.
● Collaborate with Power BI developers to prepare curated datasets and semantic models for reporting and analytics.
● Implement data quality checks, validation rules, and reconciliation processes.
● Apply security, governance, access control, and workspace management best practices.
● Troubleshoot production issues, resolve data pipeline failures, and maintain technical documentation.
● Work with Git, deployment pipelines, and CI/CD processes to support controlled releases across environments.
Required Skills
● 4–6 years of experience in data engineering, ETL development, or related data platform roles.
● Hands-on experience with Microsoft Fabric and its core data engineering components and experience in migrating Azure Synapse Analytics workloads to Microsoft Fabric is mandatory.
● Strong SQL skills, including query optimization and complex transformations.
● Experience with Python, PySpark, Spark SQL, or Apache Spark.
● Strong understanding of data warehousing, dimensional modeling, and Lakehouse architecture.
● Experience building and maintaining ETL/ELT pipelines.
● Knowledge of OneLake, Lakehouse, Warehouse, and Delta Lake.
● Familiarity with Power BI datasets and semantic models.
● Experience with data integration, data quality, and pipeline monitoring.
Preferred Skills
● Experience with Azure Data Factory, Azure Data Lake Storage, or other Azure data services.
● Knowledge of medallion architecture (Bronze, Silver, and Gold layers).
● Familiarity with Microsoft Fabric deployment pipelines, Git integration, and CI/CD.
● Understanding of data governance, security, and workspace administration.
● Microsoft Fabric or Azure data engineering certifications.
Candidate Profile
Hands-on experience in migrating Azure Synapse Analytics workloads to Microsoft Fabric is mandatory.
● Strong analytical and problem-solving skills.
● Ability to independently develop, test, and deploy data engineering solutions.
● Experience collaborating with BI developers, analysts, architects, and business stakeholders.
● Ability to troubleshoot complex data issues and deliver production-ready solutions.
Click on Apply to know more.