Gravity Infosolutions
Website:
gravityinfosolutions.com
Job details:
GenAI QA Lead
Role: GenAI QA Lead
Experience: 6–10 Years
Location: Pune / Bangalore / Hyderabad (Hybrid – 3 Days Office)
Employment Type: Full-Time
Job Description
We are looking for a GenAI QA Lead with strong experience in AI testing, automation, and evaluation frameworks for LLM-based applications. The candidate should have expertise in validating AI model outputs, RAG systems, prompts, and agent workflows.
Key Responsibilities
- Define quality metrics for LLM and Generative AI systems
- Build and maintain automated EvalOps frameworks
- Validate prompts, responses, and AI workflows
- Test RAG pipelines, grounding accuracy, and hallucination handling
- Design scalable automation frameworks for AI applications
- Integrate AI testing into CI/CD pipelines
- Analyze AI outputs and identify failure patterns
- Work closely with AI engineers and product teams
- Create test strategies, reports, and quality benchmarks
Required Skills
- Strong experience in Software QA and Automation Testing
- Experience testing AI/LLM-based applications
- Hands-on experience with DeepEval, pytest, or similar frameworks
- Experience with API testing and data validation
- Understanding of RAG systems and prompt validation
- Knowledge of CI/CD tools and Agile methodologies
- Experience with JIRA or other test management tools
- Strong analytical and problem-solving skills
Good to Have
- Exposure to LangChain, OpenAI, or AI agent workflows
- Experience with cloud platforms (AWS/Azure/GCP)
- Understanding of AI evaluation metrics and observability tools
Click on Apply to know more.