Bean HR Consulting
Website:
beanhr.com
Job details:
Job Description – Architect, Data Platform Engineering
Location: Gurugram (Hybrid)
Experience: 12+ Years
Employment Type: Full-Time
Position Summary
We are seeking an experienced Architect – Data Platform Engineering to lead the implementation and optimization of enterprise-scale data platforms on Azure and Databricks Lakehouse. This is a hands-on architecture role responsible for translating enterprise data architecture into scalable engineering solutions by designing robust data pipelines, implementing governance frameworks, building curated data models, and driving automation through CI/CD and Infrastructure as Code (IaC).
The ideal candidate will possess deep expertise in Azure Data Platform, Databricks, DevOps, and modern data engineering practices while providing technical leadership and mentoring engineering teams to deliver high-quality, scalable data solutions.
Key Responsibilities
Data Engineering & Platform Development
- Design, develop, and maintain scalable data ingestion frameworks using Azure Data Factory (ADF), Azure Databricks, PySpark, and Spark.
- Build and optimize Bronze-to-Silver transformation pipelines with integrated data quality checks, schema validation, and data contract enforcement.
- Optimize data pipelines for performance, scalability, reliability, and cost efficiency.
- Develop batch and streaming data processing solutions aligned with enterprise standards.
Data Modeling & Analytics
- Design and implement Gold-layer curated data models to support enterprise analytics and reporting.
- Develop dimensional models including Star Schema, Snowflake Schema, Fact Tables, Dimension Tables, Surrogate Keys, and Slowly Changing Dimensions (SCD).
- Ensure reusable and scalable semantic models across business domains.
- Collaborate with business stakeholders to define KPIs, business rules, and data validation scenarios.
- Manage schema evolution, backward compatibility, and impact analysis for production data models.
DevOps, Automation & Infrastructure
- Own and maintain CI/CD pipelines using Azure DevOps.
- Automate project onboarding, repository creation, deployment pipelines, and engineering templates.
- Implement Infrastructure as Code (Terraform) and automation using PowerShell and Python.
- Troubleshoot deployment issues, permissions, and pipeline failures while ensuring engineering best practices.
Data Governance & Quality
- Implement enterprise governance standards using Unity Catalog.
- Enforce metadata management, naming conventions, access controls (RBAC/ABAC), and data governance policies.
- Build and maintain data quality frameworks, schema enforcement, and metadata-driven architectures.
Architecture & Technical Leadership
- Translate enterprise architecture into detailed technical designs and reusable engineering assets.
- Develop engineering playbooks, templates, and implementation standards to drive consistency across projects.
- Promote automation-first engineering practices throughout the platform.
Team Enablement & Collaboration
- Mentor and support engineering teams through design reviews, technical guidance, and troubleshooting.
- Create reusable assets and documentation to improve delivery efficiency.
- Partner with architects, business stakeholders, and delivery teams to ensure successful implementation of enterprise data solutions.
Required Skills & Qualifications
Technical Skills
- 12+ years of experience in Data Engineering, Data Warehousing, and Enterprise Analytics Platforms.
- Strong hands-on experience with:
- Azure Data Factory (ADF)
- Azure Data Lake Storage (ADLS)
- Azure SQL
- Azure Databricks
- Delta Lake
- Delta Live Tables (DLT)
- Apache Spark / PySpark
- Databricks Workflows
- Unity Catalog
- Advanced proficiency in Python (PySpark) and SQL.
- Experience designing both batch and streaming data pipelines.
- Expertise in Azure DevOps, CI/CD pipelines, Git repositories, deployment automation, and policy management.
- Strong experience with Terraform, PowerShell, and Infrastructure as Code (IaC).
- Knowledge of Microsoft Fabric is highly preferred.
- Strong understanding of data governance, metadata management, schema enforcement, and enterprise data quality frameworks.
- Experience in performance tuning, monitoring, debugging, and production support.
Education
Required
- Bachelor's Degree in Computer Science, Information Technology, Engineering, Information Systems, or a related discipline from an accredited university.
Preferred
- Master's Degree (MIS, MBA, Computer Science, or related field).
- Microsoft Azure Data & Analytics certifications.
Preferred Industry Experience
- Experience working in Medical Devices, Healthcare, Life Sciences, or other highly regulated industries is preferred.
Soft Skills
- Excellent verbal and written communication skills.
- Strong analytical and problem-solving abilities.
- Ability to simplify complex technical concepts for business stakeholders.
- Strong stakeholder management and collaboration skills.
- Proven ability to lead technical discussions and influence engineering best practices.
- Experience mentoring engineering teams and driving technical excellence.
- Ability to identify opportunities for process optimization and continuous improvement.
Why Join Us?
- Opportunity to architect enterprise-scale Azure & Databricks data platforms.
- Work on modern cloud-native data engineering and Lakehouse technologies.
- Collaborate with cross-functional teams in delivering business-critical analytics solutions.
- Lead engineering best practices, automation initiatives, and platform innovation.
Click on Apply to know more.
