Website:
mindbrain.co.in
Job details:
Job Title: Microsoft Fabric Data Engineer (Python / PySpark)
Experience: 8+ Years
Location: India Remote
Employment Type: Contract
Job Overview
We are looking for a highly skilled Microsoft Fabric Data Engineer with strong expertise in Python, PySpark, and modern data lakehouse architectures. The ideal candidate will be responsible for building scalable data pipelines, optimizing large-scale data processing, and implementing modern data engineering practices using Microsoft Fabric and Azure ecosystem.
Key Responsibilities
• Design, develop, and maintain scalable data pipelines using Python and PySpark.
• Build and manage data ingestion frameworks supporting batch and streaming data pipelines.
• Develop and optimize Spark transformations, ensuring high performance and efficient data processing.
• Implement Lakehouse architecture using Delta Lake and Microsoft Fabric.
• Design and maintain Data Lakes and Data Warehouses for enterprise-scale analytics.
• Work with Microsoft Fabric components including Data Factory, Data Lake, Synapse, and Power BI for end-to-end data solutions.
• Develop ETL/ELT pipelines to process and transform large datasets from multiple data sources.
• Implement Medallion architecture (Bronze, Silver, Gold layers) for structured data processing.
• Optimize data pipelines for performance, scalability, and cost efficiency.
• Integrate data using APIs, replication tools, and streaming platforms.
• Build and maintain CI/CD pipelines for automated deployment of data workflows.
• Collaborate with data analysts, BI teams, and business stakeholders to deliver reliable data solutions.
• Work in Agile environments and participate in sprint planning, code reviews, and technical discussions.
Required Skills
• Strong experience in Python and PySpark for large-scale data processing.
• Hands-on experience with Microsoft Fabric platform including: Data Factory, Data Lake, Synapse, Power BI
• Strong expertise in SQL and data transformation techniques.
• Experience working with Delta Lake / Lakehouse architecture.
• Experience with Azure services such as ADF, ADLS, Azure Synapse, Azure DevOps.
• Strong understanding of distributed computing and big data processing frameworks.
• Experience implementing Medallion Architecture.
• Knowledge of data ingestion frameworks, APIs, and streaming pipelines.
• Experience with CI/CD pipelines and DevOps practices.
• Strong analytical, troubleshooting, and performance tuning skills.
Preferred Qualifications
• Experience working with large-scale enterprise data platforms.
• Familiarity with data governance, data quality, and security best practices.
• Exposure to real-time data processing and event-driven architectures.
• Certification in Azure Data Engineering or Microsoft Fabric is a plus.
Click on Apply to know more.