Intellectt Inc
Website:
intellectt.com
Job details:
Job Title: AI QA Lead
Location: Hyderabad, India
Experience: 11+ Years
Role Overview
We are looking for an experienced AI QA Lead to drive quality assurance strategies for AI/ML-driven systems. This role goes beyond traditional UI/API testing and focuses on validating machine learning models, large language models (LLMs), and data pipelines. You will play a critical role in ensuring reliability, fairness, and performance of AI-powered solutions.
Key Responsibilities
- Lead end-to-end QA strategy for AI/ML and data-driven systems
- Design and implement test frameworks for ML model validation and evaluation
- Perform LLM testing, including prompt evaluation, response validation, and bias detection
- Validate data pipelines (ETL) for data quality, consistency, and integrity
- Build and maintain automation frameworks using Python
- Define test datasets, benchmarks, and evaluation metrics for AI models
- Collaborate with data scientists, ML engineers, and product teams
- Identify risks related to model drift, bias, and performance degradation
- Implement continuous testing practices in CI/CD pipelines
- Ensure compliance with quality, ethical AI, and governance standards
Required Skills & Qualifications
- 12+ years of experience in QA, with strong exposure to AI/ML systems
- Hands-on experience in ML model testing and validation techniques
- Strong understanding of LLMs and prompt engineering/testing methodologies
- Experience with data validation and pipeline testing
- Proficiency in Python for test automation
- Knowledge of statistical analysis and model evaluation metrics
- Experience working in Agile/Scrum environments
Preferred Qualifications
- Experience with tools like Evidently AI, DeepChecks, or similar ML monitoring/testing frameworks
- Familiarity with MLOps practices and tools
- Exposure to cloud platforms (AWS, Azure, or GCP)
- Understanding of AI ethics, bias detection, and explainability
Leadership Expectations
- Mentor QA teams on AI testing practices
- Drive innovation in AI quality engineering
- Establish best practices and standards for AI testing
Click on Apply to know more.