Intellectt Inc
Website:
intellectt.com
Job details:
Job Title: AI QA Lead
Location: Hyderabad, India
Experience: 11+ Years
Role Overview:
We are looking for an experienced AI QA Lead to drive quality assurance strategies for AI/ML-driven systems. This role goes beyond traditional UI/API testing and focuses on validating machine learning models, large language models (LLMs), and data pipelines.
You will play a critical role in ensuring reliability, fairness, and performance of AI-powered solutions.
Key Responsibilities:
Lead end-to-end QA strategy for AI/ML and data-driven systems.
Design and implement test frameworks for ML model validation and evaluation.
Perform LLM testing, including prompt evaluation, response validation, and bias detection.
Validate data pipelines (ETL) for data quality, consistency, and integrity.
Build and maintain automation frameworks using Python.
Define test datasets, benchmarks, and evaluation metrics for AI models.
Collaborate with data scientists, ML engineers, and product teams.
Identify risks related to model drift, bias, and performance degradation.
Implement continuous testing practices in CI/CD pipelines.
Ensure compliance with quality, ethical AI, and governance standards.
Required Skills & Qualifications:
12+ years of experience in QA, with strong exposure to AI/ML systems.
Hands-on experience in ML model testing and validation techniques.
Strong understanding of LLMs and prompt engineering/testing methodologies.
Experience with data validation and pipeline testing.
Proficiency in Python for test automation.
Knowledge of statistical analysis and model evaluation metrics.
Experience working in Agile/Scrum environments.
Preferred Qualifications:
Experience with tools like Evidently AI, DeepChecks, or similar ML monitoring/testing frameworks.
Familiarity with MLOps practices and tools.
Exposure to cloud platforms (AWS, Azure, or GCP).
Understanding of AI ethics, bias detection, and explainability.
Leadership Expectations:
Mentor QA teams on AI testing practices.
Drive innovation in AI quality engineering.
Establish best practices and standards for AI testing.
Click on Apply to know more.