e-Zest Solutions
Website:
e-zest.com
Job details:
Company Description e-Zest, an Accion Labs Company, is a digital innovation partner that helps enterprises and technology companies leverage emerging technologies and design to create impactful customer experiences. The organization is strongly customer-focused and technology-driven, enabling clients to derive measurable business value from their digital transformation initiatives. Since 2000, e-Zest has consistently delivered innovative solutions across sectors such as technology, healthcare, travel, engineering, and international organizations. Team members work on modern, cloud-based and data-centric projects that emphasize quality, collaboration, and continuous learning. The company offers an environment where professionals can advance their expertise while contributing to complex, global digital transformation programs.
Role Description This is a full-time, hybrid role for a Senior Azure Databricks Solution Architect and Platform Administrator, based in the Pune District with flexibility for partial work from home. The role involves architecting scalable data and analytics solutions on Azure, with a focus on Azure Databricks, Delta Lake, and related Azure data services. Responsibilities include designing end-to-end data platforms, defining data ingestion and transformation patterns, and ensuring performance, security, and reliability of the Databricks environment. The professional in this role will administer Azure Databricks workspaces and clusters, manage user access and governance, optimize costs, and implement best practices for CI/CD and environment automation. Day-to-day work includes collaborating with cross-functional teams, guiding development and data engineering teams, conducting technical reviews, troubleshooting complex platform issues, and mentoring colleagues on Azure and Databricks standards.Key Tasks and Deliverables
- Collaborate with Data Platforms and Integration team in architecting and implementing scalable enterprise data platform solutions utilizing Azure Databricks, Delta Lake, Unity Catalog, Azure Data Lake Storage Gen2, Azure Data Factory, Azure Event Hub, Azure SQL, and related Azure services.
- Design and develop enterprise data engineering frameworks supporting batch, streaming, ELT/ETL, and real-time analytics workloads.
- Administer and maintain Azure Databricks workspaces including cluster configuration, job scheduling, workspace administration, Unity Catalog implementation, secret management, compute policies, cluster policies, and workspace governance.
- Define and implement enterprise security architecture including identity integration with Microsoft Entra ID (Azure AD), Role-Based Access Control (RBAC), Privileged Identity Management (PIM), data encryption, private networking, managed identities, Key Vault integration, and secure access patterns.
- Establish enterprise data governance standards utilizing Unity Catalog including data lineage, data quality, metadata management, catalog organization, access policies, and audit capabilities.
- Deploy and Configure CI/CD pipelines utilizing Azure DevOps and Databricks Asset Bundles (DAB), Infrastructure as Code (Terraform/Bicep where applicable), deployment automation, release management, and environment promotion strategies.
- Optimize Databricks workloads through performance tuning, Spark optimization, cluster sizing, workload isolation, caching strategies, partitioning, Delta optimization, Photon, and cost optimization best practices.
- Design enterprise monitoring, logging, operational dashboards, alerting, and platform health monitoring utilizing Azure Monitor, Log Analytics, Application Insights, and Databricks system tables.
- Collaborate with business analysts, solution architects, data engineers, and application teams to analyze business requirements and translate them into enterprise-grade data platform solutions.
- Develop reusable architecture patterns, reference implementations, development standards, naming conventions, coding standards, operational procedures, and platform documentation.
- Support migration of legacy enterprise data warehouses, ETL solutions, and reporting platforms to Azure Databricks and modern Lakehouse architecture.
- Produce comprehensive technical documentation including solution designs, operational runbooks, deployment guides, architecture diagrams, security configurations, and knowledge transfer documentation.
- Provide technical support and troubleshooting for production incidents related to Azure Databricks, Spark workloads, data pipelines, and platform infrastructure.
ADDITIONAL INFORMATION WHEN APPLICABLE
(This field should contain information necessary to enable the LTAS holder to provide a complete proposal: It may include (1) Description of current UNICEF technical environment; (2) Dependencies, if any, etc.)
- University degree in Computer Science, Information Systems, Engineering, or a related discipline.
- Minimum 8+ years of experience designing and implementing enterprise data platforms.
- Minimum 5+ years of hands-on Azure Databricks experience in enterprise environments.
- Strong experience architecting modern Lakehouse solutions using Azure Databricks and Delta Lake.
- Strong experience administering Azure Databricks workspaces, clusters, Unity Catalog, compute policies, job orchestration, and workspace security.
- Extensive experience developing data engineering solutions using Apache Spark (PySpark and/or Spark SQL).
- Strong experience implementing enterprise data governance utilizing Unity Catalog, metadata management, data lineage, and fine-grained access controls.
- Strong knowledge of Azure Data Lake Storage Gen2, Azure Data Factory, Azure Event Hub, Azure SQL Database, Azure Key Vault, Azure Monitor, Microsoft Entra ID (Azure AD), and Azure networking.
- Experience implementing Infrastructure as Code using Terraform and/or Bicep.
- Strong experience implementing CI/CD pipelines utilizing Azure DevOps, automated testing, release management, and deployment automation.
- Experience designing and working with enterprise security architectures including RBAC, Managed Identities, Private Endpoints, Key Vault integration, encryption, and network security.
- Strong understanding of data modelling techniques including dimensional modelling, Data Vault, medallion architecture, and enterprise data warehouse concepts.
- Experience implementing enterprise data quality frameworks, monitoring, observability, and operational dashboards.
- Experience with Power BI, Microsoft Fabric, Synapse Analytics, or other enterprise analytics platforms is highly desirable.
- Knowledge of AI/ML workloads, Databricks ML, Feature Store, Model Serving, and Generative AI architectures is considered an asset.
- Strong SQL programming skills.
- Proficiency in Python and PySpark development.
- Excellent analytical, problem-solving, documentation, and communication skills.
- Experience working within large international organizations, NGOs, or United Nations agencies is desirable.
- The following certifications are desirable:
- Databricks Certified Platform Architecture
- Microsoft Azure Solution Architecture Expert
- Databricks Certified Data Engineer Associate/Professional
Click on Apply to know more.