Dhruva Space
Website:
dhruvaspace.com
Job details:
Role Overview and Responsibilities
Dhruva Space is seeking an experienced Lead Quality Engineering & AI Evaluation to own software quality, QA/QC, AI evaluation, monitoring, and reliability across the organization’s engineering teams. The role will build the quality and AI evaluation function from the ground up, establish scalable QA/QC practices, develop evaluation frameworks for AI agents, and ensure reliable performance of AI-powered systems in production. The role will also lead and mentor the quality and evaluation team while working closely with engineering and leadership teams.
Key responsibilities include but are not limited to:
- Define and own the software QA/QC strategy across all engineering teams, covering both traditional and AI-agent-generated code.
- Design, build, and own evaluation frameworks for measuring AI agent accuracy, reliability, hallucination, drift, performance, and cost.
- Establish production monitoring, observability, alerting, and escalation processes for AI agent performance.
- Build and lead a quality and AI evaluation team, including hiring, mentoring, and developing team members.
- Define quality gates, testing standards, and release criteria for software and AI-agent-generated code.
- Establish review and testing standards for AI-assisted and AI-generated software development.
- Define and track software quality and AI agent performance metrics and report them to the leadership team.
- Investigate systemic quality issues and drive root-cause analysis and corrective actions across engineering teams.
- Establish feedback loops between evaluation results, production monitoring, agent performance, and prompt/model improvements.
- Collaborate with engineering, product, AI/ML, and architecture teams to improve software and AI system quality.
- Act as the primary escalation point for critical software quality and AI agent performance issues in production.
- Build scalable QA/QC and AI evaluation practices that can support multiple products and AI agents.
- Establish processes and standards that ensure consistent quality as the engineering and AI ecosystem scales.
Candidate Requirements:
- Bachelor’s or Master’s degree in Computer Science, Software Engineering, AI/ML, Data Science, or a related field.
- 8–12 years of experience in QA/QC, Quality Engineering, Software Quality, AI/ML evaluation, or related roles.
- Proven experience leading or building a QA/QC or Quality Engineering function, preferably from the ground up.
- Strong practical experience designing evaluation frameworks for ML/LLM or AI-agent systems.
- Strong understanding of AI agent architectures and evaluation metrics such as accuracy, hallucination, reliability, drift, latency, and cost.
- Strong understanding of software testing, automation, CI/CD, release management, and quality gates.
- Experience with AI evaluation and observability tools such as DeepEval, LangSmith, Arize Phoenix, or equivalent.
- Experience designing production monitoring, alerting, and observability for AI/ML systems.
- Proven experience building, mentoring, and managing quality or engineering teams.
- Experience in safety-critical, regulated, aerospace, defence, or high-compliance software environments is preferred.
Click on Apply to know more.
