Omniskope
Website:
omniskope.com
Job details:
Job Title: Databricks Engineer
Location: India (Remote)
Primary Skills: Databricks Agent Bricks, Databricks, Python, SQL, PySpark, Generative AI, LLMs, RAG, Mosaic AI, MLflow, Vector Search, REST APIs
Job Description
We are looking for a Databricks AI Engineer to design, build and run production AI agents natively on the Databricks platform for a client engagement. You will turn the client’s governed lakehouse data into reliable, secure AI assistants that business users trust. You will own the full agent lifecycle, from data preparation and retrieval through deployment, evaluation and ongoing improvement.
This is a hands-on engineering role for someone who has already shipped agents on Databricks, not someone looking to learn the platform on the job.
Responsibilities:
- Design and build AI agents using the Databricks Mosaic AI Agent Framework or Agent Bricks, and deploy them on Databricks Model Serving for business users.
- Build retrieval pipelines on Databricks Vector Search, covering document parsing, chunking, embeddings and indexing, with indexes kept in sync with Delta tables.
- Implement agent tools and data access through Unity Catalog, so agents respect user permissions using row filters, column masks and service principals.
- Trace, evaluate and improve agent quality with MLflow, tracking groundedness, correctness, latency and cost.
- Build and maintain the PySpark and Delta Lake pipelines that prepare clean data and documents for agents.
- Deploy agents, pipelines and serving endpoints through CI/CD using Databricks Asset Bundles or equivalent, with proper environment separation, versioning and rollback.
- Work with client architects and business stakeholders to turn broad requests into well-defined agent use cases with clear success criteria.
- Explain agent design, limitations and risks clearly to technical and non-technical audiences.
- Document solutions so the client team can operate and extend them.
Required skills:
- Hands-on production experience building and deploying AI agents on Databricks with Mosaic AI Agent Framework or Agent Bricks, Model Serving, Vector Search, Unity Catalog and MLflow. You should be able to walk through specific projects in detail.
- Strong Python and PySpark skills on the Databricks lakehouse, including Delta Lake.
- Experience with agent design patterns such as tool calling, routing, state management, structured outputs, guardrails and RAG.
- Experience with CI/CD and production operations for data and AI workloads.
- Experience in a client-facing consulting or professional-services environment.
- Clear communication with both technical and business stakeholders.
Preferred
- Databricks Generative AI Engineer Associate or Data Engineer Professional certification.
- Experience with LangChain or LangGraph integrated with Databricks.
- Experience in regulated industries where data privacy and auditability matter.
- Day-to-day use of AI tools to speed up coding, testing and documentation.
Click on Apply to know more.