Softkode Technologies
Website:
softkode.io
Job details:
🚀 We’re Hiring: Azure Data Engineer(Azure + Python + Pyspark)
📍 Location: India
💼 Employment Type: Full-Time
🧑💻 Experience: 4+
We are looking for a skilled Azure Data Engineer to join our team and work on the Mahindra Finance Project. The ideal candidate will design, build, and maintain scalable data pipelines for financial data processing using Azure cloud technologies and Databricks.
🔹 Key Responsibilities
Data Pipeline Development
• Design and develop ETL/ELT pipelines using Azure Data Factory (ADF)
• Ingest data from multiple source systems into Azure Data Lake Storage (ADLS Gen2)
• Automate daily batch ingestion pipelines with proper orchestration
Data Processing & Transformation
• Implement transformations using PySpark and Spark SQL in Azure Databricks
• Convert existing SQL Server logic into Spark-based processing
• Build and maintain Delta Lake tables using Medallion Architecture (Bronze, Silver, Gold)
Data Modeling & Data Quality
• Design optimized data models for analytics and reporting
• Implement validation checks, reconciliation processes, and data quality frameworks
• Ensure data accuracy and consistency across data layers
Performance Optimization
• Optimize Spark jobs and Databricks workloads
• Implement partitioning, caching, and efficient joins
• Improve pipeline performance and reduce processing time
Data Governance & Security
• Manage secrets securely using Azure Key Vault
• Implement RBAC and managed identities for secure access
• Set up logging, monitoring, and alerts for pipelines
Reporting Layer Support
• Build Gold layer tables and views for Power BI dashboards
• Collaborate with analytics teams to support reporting needs
🔹 Required Technical Skills
✔ Azure Data Factory
✔ Azure Databricks
✔ Azure Data Lake Storage Gen2
✔ Azure Key Vault
✔ PySpark, Spark SQL
✔ Python, SQL
✔ Delta Lake & Medallion Architecture
✔ Data pipeline orchestration & performance optimization
👉 Interested candidates can apply or send their resume :-krishnasharma@softkode.io
Click on Apply to know more.