Website:
nam-info.com
Job details:
Role : Azure Databricks Engineer
Location : Bannerghatta Mainroad. Near Jayadeva ( After Virtual technical interview F-2-F or IN Person interview is a must)
Duration : FTE
Key Responsibilities
- Design, build, and maintain scalable ETL/ELT pipelines in Databricks (Delta Live Tables, Workflows, notebooks/jobs) ingesting data from operational systems, APIs, and files
- Implement medallion architecture (bronze/silver/gold) data models on Delta Lake, optimized for reliability and query performance
- Write efficient PySpark, Spark SQL, and Python for large-scale transformation and processing jobs
- Implement data quality checks, expectations (e.g. Delta Live Tables expectations / Great Expectations), and pipeline monitoring and alerting
- Optimize cluster configuration, job performance, and storage costs (partitioning, Z-ordering, Photon, auto-scaling)
- Manage code and pipeline deployment through Databricks Repos, Git, and CI/CD (Databricks Asset Bundles or equivalent)
- Collaborate with analysts, BI, and application teams to translate data needs into robust pipeline design
- Maintain documentation for lineage, schemas, and pipeline logic in Unity Catalog
- Participate in code reviews, on-call rotation, and platform architecture discussions
Required Qualifications
- 7–8 years of hands-on data engineering experience
- Production experience building pipelines on Databricks, including Delta Lake, Spark (batch and structured streaming), and Databricks Workflows or Delta Live Tables
- Strong PySpark and Spark SQL skills, plus general Python proficiency
- Solid understanding of Lakehouse/medallion data modeling and dimensional modeling concepts
- Experience with Unity Catalog or similar governance/cataloging tooling (access control, lineage, data discovery), Genie expertise.
- Familiarity with orchestration (Databricks Workflows, Airflow, or dbt) and CI/CD for data pipelines
- Working knowledge of cloud infrastructure underlying Databricks (Azure) — storage, networking, IAM basics
- Understanding of data governance, security, and privacy practices (RBAC, PII handling, encryption)
Preferred / Nice to Have
- Databricks certification (Data Engineer Associate/Professional)
- Experience with Genie One, Genie code and Dataricks Marketplace
- Exposure to real estate, property management, or SaaS enterprise data
- Experience with infrastructure-as-code (Terraform) for Databricks workspace/cluster provisioning
- Prior experience contributing to a greenfield enterprise data platform build
Manjunath
Staffing Manager
+91 9148218735 | manju@nam-info.com
www.nam-info.com
#29/2B-01, 1st Floor, K.R. Road
Banashankari 2nd Stage, Bengaluru - 560070
USA | CANADA | INDIA
Click on Apply to know more.
