Website:
cogniify.ai
Job details:
Role : Databricks Architect
Panindia - Multiple locations- hybrid at Hyderabad, Pune, Bengaluru, Chennai, Goa, Gurugram, etc..
Fulltime with Cogniify
Please provide the following details along with an updated resume at parveza@cogniify.ai
Skills:
Databricks:
Python. pyspark coding:
Advanced SQL:
The Role
We're seeking a Databricks Architect (Senior/Specialist level) to serve as a senior technical authority for data architecture and analytics engineering on the Databricks Lakehouse platform across Cognify Analytics and our client engagements. In this role, you will design and own Databricks-based data platform architecture, lead complex data engineering initiatives, and ensure our data capabilities are enterprise-grade, governed, and future-ready. You will bridge data engineering, analytics, and AI/ML — combining deep hands-on expertise in Databricks, Python, and SQL with strong architectural judgment and stakeholder communication.
Must-Have Skills (Non-Negotiable)
- Databricks: hands-on architecture and engineering experience on the Databricks Lakehouse Platform (production-grade, at scale)
- Python: strong professional proficiency for data engineering and pipeline development
- SQL: advanced proficiency for data modeling, transformation, and performance tuning
Candidates without demonstrable, hands-on experience in all three of the above will not be considered.
What You'll Do
- Own the architecture and design of Databricks-based data platforms, including lakehouse design (Delta Lake), medallion architecture, and unified analytics layers.
- Serve as the senior technical authority on Databricks platform design, providing guidance on architecture, tooling, modeling, and engineering standards across teams and projects.
- Design and build ingestion, transformation, and orchestration pipelines using Python, SQL, PySpark, and Databricks Workflows.
- Architect data mesh and data product strategies, defining domain ownership, data contracts, and self-service consumption patterns on Databricks.
- Establish and evangelize best practices across the data lifecycle: ingestion, transformation, modeling, quality, observability, governance, and consumption.
- Drive the integration of Databricks with AI/ML capabilities, including feature engineering pipelines, vector data infrastructure for RAG, and MLflow-based model workflows.
- Lead complex data migration, platform modernization, and consolidation initiatives for enterprise clients across industries.
- Evaluate emerging data technologies (Apache Iceberg, Unity Catalog, Delta Live Tables, Mosaic AI) to inform architectural decisions.
- Solve complex, ambiguous, high-impact data architecture problems that span multiple teams, platforms, or organizational boundaries.
- Drive cross-functional alignment between data engineering, analytics, AI/ML, platform engineering, security, and product teams.
- Mentor senior data engineers and analysts, fostering a culture of technical excellence.
- Represent Cognify Analytics in discussions with client stakeholders and leadership on data capabilities, strategy, and technical roadmaps.
- Define and enforce data governance, compliance (GDPR, HIPAA, SOC2), and responsible data management standards across all Databricks environments.
- Drive FinOps maturity for the Databricks platform, including compute optimization, cluster policies, storage lifecycle management, and cost forecasting.
What We're Looking For
- Bachelor's, Master's, or equivalent professional experience in Computer Science, Data Science, Statistics, or a related field.
- 6–9 years of professional experience in data engineering, analytics engineering, or data architecture, with demonstrated technical leadership on at least a few large-scale engagements.
- Mandatory, hands-on expertise in Databricks, Python, and SQL in production environments.
- Strong working knowledge of the modern data stack: dbt, Airflow/Dagster, Fivetran/Airbyte, Spark, Kafka, and cloud-native data services.
- Solid grasp of data modeling methodologies (Kimball, Data Vault, Activity Schema, OBT) and the judgment to apply them across contexts.
- Experience with cloud data infrastructure on AWS, Azure, or GCP (any combination is acceptable, alongside Databricks).
- Proven ability to influence technical direction and align technical and business stakeholders on complex architecture topics.
- Strong understanding of data governance, data quality, cataloging, lineage, and regulatory compliance frameworks.
- Experience mentoring engineers and contributing to a high-performing data team.
- Strong understanding of how data platforms serve AI/ML workloads, including feature engineering, vector data, and model input/output pipelines.
Preferred Qualifications
- Experience architecting Databricks-based platforms that directly serve LLM-based systems, RAG pipelines, and agentic AI architectures at enterprise scale.
- Deep familiarity with lakehouse table formats: Delta Lake, Apache Iceberg, and Apache Hudi.
- Track record of implementing data mesh, data product, or federated data governance patterns.
- Experience with real-time analytics and streaming architectures: Kafka, Flink, Spark Structured Streaming.
- Experience with advanced Databricks features: Unity Catalog, Delta Live Tables, Databricks Workflows, MLflow, and Mosaic AI.
- Contributions to open-source data projects, data architecture publications, or industry standards bodies.
- Background in financial services, healthcare, SaaS, or enterprise consulting requiring high data compliance, security, and operational rigor.
- Experience leading data engineering across geographically distributed teams and multi-client engagements
Click on Apply to know more.
