CoE - AI - AI Test Lead

Qualitest

Website: qualitestgroup.com
Job details:
We are seeking an AI Test Lead / Senior Test Engineer for test planning and end-to-end testing of cloud-based, data-driven AI Ops platform. The role requires strong expertise in testing, automation, AI/ML, data validation, and cloud services (AWS preferred). He/she will play a pivotal role in ensuring data quality, workflow reliability, automation robustness and AI model accuracy, working across engineering, data science and operations teams. Key Responsibilities: Test planning, test case development and test execution covering data ingestion, pipelines, workflows, AI models and cloud deployments.Establish testing methodologies for data quality, workflow orchestration, model validation, and system integrations.Drive adoption of automation-first testing approaches across CI/CD pipelines.Define validation frameworks for structured, semi-structured, and unstructured data.Implement checks for data integrity, transformations, schema compliance, lineage, and reconciliation.Test workflow orchestration (e.g., Airflow, Step Functions, Lambda, EventBridge).Collaborate with data science teams to design model validation tests (accuracy, bias, drift, fairness, performance).Automate AI model regression testing and monitoring in production environments.Architect test automation frameworks for data and AI pipelines.Build synthetic data generation strategies for testing AI/ML use cases.Integrate testing into DevOps/MLOps pipelines for continuous quality validation.Leverage AWS services (S3, Glue, Athena, Redshift, SageMaker, Step Functions, Lambda, CloudWatch, etc.) in testing frameworks.Ensure performance, scalability, security, and compliance testing of cloud-native solutions.Design failover, recovery, and resilience tests for critical workflows.Define quality KPIs and dashboards for data and AI testing outcomes. Key Skills & Qualifications: Strong foundation in software testing principles, test architecture, and automation frameworks.Hands-on experience in data testing, ETL/ELT pipelines, and big data platforms.Understanding of AI/ML lifecycle including training, validation, deployment, and monitoring.Experience in testing AI/ML models (accuracy, drift, fairness, explainability).Proficiency in Python, PyTest, Robot Framework, or similar automation tools.Familiarity with data quality tools (e.g., Great Expectations, Soda, Deequ).Experience with AWS cloud services (S3, Glue, Lambda, SageMaker, Athena, Redshift, Step Functions).Strong problem-solving and analytical thinking.Excellent communication and stakeholder management skills.3 Must haves:Python 4/5ETL (Data) testing 4/5Test Automation framework development 4/5 AI/ML 4/5 Click on Apply to know more.