zealant consulting group
Website:
zealantgroup.com
Job details:
Position Overview
Job Title: Engineering Manager Data Engineering
Location: Pune, Maharashtra, India
Work Mode: Hybrid
Experience Required: 10-15 Years
Open Positions: 1
Job Summary
We are seeking an experienced and dynamic
Engineering Manager Data Engineering to lead cross-functional data engineering teams and drive the design, development, and delivery of enterprise-scale data platforms. This is a
50-50 blend of technical leadership and delivery management role where you will combine hands-on technical expertise with strategic people leadership. You will be responsible for leading multiple squads (20+ members), ensuring exceptional delivery quality, and building scalable data solutions using Databricks on cloud platforms. The ideal candidate will have a proven track record of end-to-end delivery, strong stakeholder management capabilities, and deep expertise in modern data engineering practices, particularly in the Banking/Financial Services sector.
Key ResponsibilitiesTechnical Leadership & Architecture (50%)
- Lead the design and implementation of enterprise-scale data platforms using Databricks as the primary technology stack.
- Define and enforce technical architecture, coding standards, and engineering best practices across all data engineering initiatives.
- Drive adoption of modern data engineering patterns including Medallion Architecture, Delta Lake, and Delta Live Tables (DLT).
- Conduct comprehensive solution design reviews and provide technical guidance on complex data engineering challenges.
- Review code quality, performance optimization strategies, and ensure adherence to data security, governance, and compliance standards.
- Mentor team members on advanced Databricks features, PySpark optimization, and cloud-native data architectures.
- Stay current with emerging technologies and industry trends; champion innovation within the team.
- Ensure implementation of robust data modeling strategies (Star Schema, Snowflake, Data Vault) and ETL/ELT design patterns.
Delivery Management & Execution (50%)
- Own end-to-end delivery accountability for multiple data engineering initiatives, from requirements gathering through production deployment and post-go-live support.
- Lead and manage 2+ squads (20+ members) with a strong focus on delivery excellence and quality outcomes.
- Develop comprehensive project plans, timelines, and resource allocations with accurate estimation, budgeting, and costing.
- Execute sprint planning and management; track progress against milestones and KPIs.
- Proactively identify, assess, and mitigate delivery risks and dependencies.
- Ensure production readiness through rigorous testing, validation, and release management processes.
- Drive continuous improvement in engineering processes, tools, and team productivity.
- Manage stakeholder expectations, communicate progress transparently, and prioritize business requirements effectively.
People Leadership & Development
- Lead, mentor, and develop a high-performing team of Data Engineers and Technical Leads.
- Conduct regular one-on-one meetings, performance reviews, and career development discussions.
- Support hiring initiatives, onboarding programs, and capability development plans.
- Foster a culture of collaboration, innovation, accountability, and continuous learning.
- Identify and nurture talent; create pathways for career growth within the team.
Operational Excellence & Stakeholder Management
- Ensure platform reliability, availability, and optimal performance in production environments.
- Drive root cause analysis for production incidents and implement preventive measures.
- Improve monitoring, alerting, observability, and incident response capabilities.
- Optimize cloud infrastructure costs and resource utilization across Azure, AWS, or GCP.
- Collaborate effectively with Product Owners, Solution Architects, Business stakeholders, and Platform teams.
- Communicate technical decisions, risks, and delivery status to senior leadership and business partners.
Required Skills & CompetenciesDatabricks & Data Engineering (Must-Have)
- Databricks Expertise: Databricks Workspace, Delta Lake, Delta Live Tables (DLT), Unity Catalog, Databricks Workflows, Auto Loader, Structured Streaming
- Programming: Python, PySpark, Spark SQL (SQL is mandatory; Python/PySpark is good-to-have)
- Data Engineering Patterns: ETL/ELT design, Medallion Architecture, Batch and Streaming data pipelines
- Data Modeling: Star Schema, Snowflake Schema, Data Vault methodologies
- Database Technologies: SQL Server, Oracle, Snowflake, PostgreSQL
Cloud Platforms (Must-Have)
- Hands-on experience with Microsoft Azure (primary) and/or AWS
- Cloud Storage: ADLS (Azure Data Lake Storage), S3, or GCS
- Understanding of cloud-native architectures and infrastructure optimization
DevOps & CI/CD
- Git and version control best practices
- Azure DevOps or GitHub for CI/CD pipeline management
- Infrastructure as Code (Terraform preferred)
- Release management and deployment automation
Leadership & Delivery Management (Must-Have)
- Engineering leadership with proven experience leading multiple squads (20+ members)
- Agile delivery and project management expertise
- Strong planning, estimation, budgeting, and costing skills
- Stakeholder management and executive communication
- Risk management and conflict resolution
- Coaching, mentoring, and team development capabilities
- Solution and design thinking with end-to-end delivery ownership
- Excellent communication and interpersonal skills
Domain Expertise (Preferred)
- Banking/Financial Services industry experience (highly preferred)
- Understanding of financial data governance, compliance (GDPR, SOX), and security requirements
Experience Requirements
- Total Experience: 10-15+ years in Data Engineering and related roles
- Leadership Experience: 3-5+ years leading engineering teams or squads
- Databricks Experience: Hands-on, production-grade experience with Databricks on Azure, AWS, or GCP (mandatory)
- Enterprise-Scale Platform Development: Proven experience building and delivering enterprise-scale data platforms
- End-to-End Delivery: Demonstrated ability to own complete delivery lifecycle from requirements gathering, design, development, testing, production deployment, and post-go-live support
- Performance Tuning & Optimization: Experience optimizing data pipeline performance, cost management, and production support
- Solution Architecture: Strong background in designing and implementing complex data solutions
Preferred Certifications & Qualifications
- Databricks Certified Data Engineer Professional
- Azure Data Engineer Associate or Azure Solutions Architect Expert
- AWS Certified Data Analytics Specialty
- Bachelor's degree in Computer Science, Engineering, or related field
Nice-to-Have Skills
- MLflow and ML pipeline orchestration
- Databricks Asset Bundles (DABs)
- Apache Kafka or Azure Event Hubs experience
- AI/ML pipelines and Generative AI knowledge
- Data Mesh or Data Fabric architecture experience
- Databricks Genie or AI-assisted development tools
- NoSQL databases (MongoDB, Cassandra)
- Advanced monitoring and observability tools
Key Competencies
- Technical Expertise: Deep hands-on knowledge of modern data engineering and cloud technologies
- Leadership: Ability to inspire and lead high-performing teams toward ambitious goals
- Delivery Excellence: Demonstrated track record of on-time, quality delivery of complex projects
- Strategic Thinking: Capability to align technical solutions with business objectives
- Communication: Exceptional verbal and written communication skills; ability to articulate complex concepts to diverse audiences
- Problem-Solving: Strong analytical and troubleshooting capabilities
- Adaptability: Comfortable in fast-paced, dynamic environments with evolving requirements
- Accountability: Takes ownership of outcomes and drives results
What We're Looking For
- A hands-on technical leader who can code and architect solutions while managing teams
- Someone with proven expertise in Databricks and enterprise data platform development
- A delivery-focused professional with strong planning and estimation capabilities
- A leader who has successfully managed multiple squads and driven complex, multi-initiative programs
- Banking/Financial Services industry experience is a significant plus
- An individual with excellent communication skills who can engage effectively with all stakeholder levels
- A champion of quality, continuous improvement, and operational excellence
Skills: enterprise data engineering,microsoft azure (adls, azure data factory, azure devops),data platform,databricks,delta lake & delta live tables (dlt),pyspark,engineering manager,medallion architecture
Click on Apply to know more.