Sr Test Analyst(Gen AI Platform/LLM testing)
Innodata Inc.
- Location
- Noida, Uttar Pradesh, India
- Job type
- Full-time
Required skills
- Agile
- AWS
- API
- Azure
- Cypress
- end-to-end
- GitHub
- Jenkins
- Jira
- Root Cause Analysis
- TypeScript
About the role
Innodata Inc.
Website:
innodata.com
Job details:
Key Responsibilities
- Quality Strategy Ownership: Define the end-to-end testing strategy for GenAI applications(e.g., summarization, conversational flows, content generation), including selecting appropriate methodologies for agentic AI systems.
- Mentorship & Team Guidance: Lead and mentor junior QA team members, providing direction on AI-specific testing crafts, prompt engineering, and automation frameworks.
- LLM Evaluation: Validate model outputs for factual accuracy, tone, relevance, and consistency.
- Prompt Engineering & Validation: Test and refine prompt-response workflows to identify hallucinations and edge cases
- Performance Engineering: Evaluate model latency, throughput, token usage, and API response times.
Required Skills & Experience
- Experience: 5-8 years in software testing, with 13 years specialized in testing GenAI/LLM models.
- GenAI Tools: Experience with LLMs (e.g., OpenAI GPT, Claude)
- Cloud Platforms: Familiarity with AWS (Bedrock), Azure (OpenAI) etc.
- Automation Tools: Knowledge of API testing tools (Postman, REST Assured) and CI/CD tools (Jenkins, GitHub Actions), Cypress with typescript.
- QA Fundamentals: Strong understanding of Agile methodologies, functional testing, and defect tracking (Jira)
- Root Cause Analysis (RCA): Perform deep-dive analyses of production failures in AI systems to differentiate between code bugs and model behavioural issues
Click on Apply to know more.
This page is fully interactive when JavaScript is enabled. Please enable JavaScript to apply or browse related roles.