Data Engineer

Data Engineer
 

Location: Bangalore 
Experience: 3-5 years

About Us:
VNB Consulting Services, headquartered in New Jersey, is an end-to-end technology solution provider specializing in Apps, Automation, Analytics, and AI. Our mission is to deliver tested technology solutions that enhance existing technology, improve business processes, or develop new solutions.

Role Overview:
As a Microsoft Fabric Data Engineer, you will play a critical role in designing, implementing, and supporting data warehousing and business intelligence solutions. You will work with Fabric Data flow and Data pipelines, PySpark notebooks, Spark SQL, Azure Data Factory, and Python to build scalable and efficient data pipelines, ensuring seamless data ingestion, transformation, and loading processes. This role also involves troubleshooting and enhancing performance for large-scale data sets.

Key Responsibilities:

• Design and implement scalable and efficient data pipelines using Microsoft Fabric technologies (Dataflow gen2, Data Pipelines, PySpark notebooks, Spark SQL, and Python).
• Execute data ingestion, transformation, and loading processes for both structured and unstructured data from diverse sources.
• Develop and maintain ELT (Extract, Load, Transform) processes for efficient data transformation and integration into data warehouses or analytical systems.
• Hands-on development in Microsoft Fabric, Azure Synapse, Azure SQL, Databricks, and Azure Data Lake.
• Write and optimize SQL queries on Microsoft Azure Synapse Analytics, leveraging dedicated and serverless resources.
• Perform deep code-level analysis and troubleshooting of Spark core internals, Spark SQL, Structured Streaming, and Delta.
• Monitor, troubleshoot, and fine-tune data pipelines and processing workflows to enhance performance and efficiency, especially with large data sets.
• Ensure security and compliance with data privacy regulations throughout the data engineering lifecycle.
• Build and maintain infrastructure for optimal data

Nice to have:

• Experience or knowledge of Power BI, Azure Analysis Services 
• CI/CD of data pipelines
• Microsoft Certifications (AZ-900, DP-203, DP-600)
• Working knowledge of multiple database technologies
• Experience working in agile methodology using either Azure DevOps or any similar tool

Why Join VNB Consulting Services? 

• Excellent pay and benefits 
• Flexible working schedule 
• Opportunity to work with cutting-edge Microsoft Fabric and Power BI technologies

Candidates must apply with the latest copy of their resume.