PwC
Website:
pwc.com
Job details:
At PwC, our people in software and product innovation focus on developing cutting-edge software solutions and driving product innovation to meet the evolving needs of clients. These individuals combine technical experience with creative thinking to deliver innovative software products and solutions. In testing and quality assurance at PwC, you will focus on the process of evaluating a system or software application to identify any defects, errors, or gaps in its functionality. Working in this area, you will execute various test cases and scenarios to validate that the system meets the specified requirements and performs as expected.
Focused on relationships, you are building meaningful client connections, and learning how to manage and inspire others. Navigating increasingly complex situations, you are growing your personal brand, deepening technical expertise and awareness of your strengths. You are expected to anticipate the needs of your teams and clients, and to deliver quality. Embracing increased ambiguity, you are comfortable when the path forward isn’t clear, you ask questions, and you use these moments as opportunities to grow.
Skills
Examples of the skills, knowledge, and experiences you need to lead and deliver value at this level include but are not limited to:
- Respond effectively to the diverse perspectives, needs, and feelings of others.
- Use a broad range of tools, methodologies and techniques to generate new ideas and solve problems.
- Use critical thinking to break down complex concepts.
- Understand the broader objectives of your project or role and how your work fits into the overall strategy.
- Develop a deeper understanding of the business context and how it is changing.
- Use reflection to develop self awareness, enhance strengths and address development areas.
- Interpret data to inform insights and recommendations.
- Uphold and reinforce professional and technical standards (e.g. refer to specific PwC tax and audit guidance), the Firm's code of conduct, and independence requirements.
Job Summary
We are looking for a
GenAI / LLM Agent Test Engineer to assure quality, reliability, and responsible behavior of
LLM‑based and RAG‑based agents, including
single‑agent and multi‑agent workflows. The role focuses on
prompt engineering, hallucination detection, agent reasoning validation, and automated evaluation of GenAI outputs using modern LLM testing frameworks.
Perform functional, regression, integration, and system testing for AI-driven workflows.
Test edge cases, hallucination scenarios, and adversarial prompts.
Key Responsibilities
- Test and validate LLM‑based and RAG‑based agents, including reasoning, memory, and decision‑making behavior
- Design and execute prompt engineering and adversarial testing to uncover hallucinations, bias, and edge cases
- Perform hallucination testing, relevance checks, and factual accuracy validation
- Implement unit tests for LLMs using LLM‑as‑a‑Judge techniques
- Validate agent reasoning chains, memory states, and conversation persistence
- Embed LLMs within testing pipelines to:
- Detect hallucinations
- Run adversarial prompt testing
- Perform automated evaluation of agent outputs
- Ensure AI Ethics and Responsible AI compliance in testing coverage
- Collaborate with platform, data, and GenAI teams for continuous quality improvement
Qualifications And Skills
- Bachelor’s or master’s degree in computer science, Engineering, Data Science, or equivalent experience.
- 6+ years of experience in QA automation or software engineering.
- 3+ years of experience working with AI‑driven automation or intelligent testing frameworks.
- Minimum 2 years of hands‑on experience in building or testing AI‑based tools or platforms.
Must Have Tools
- LangChain + LangSmith
- DeepEval
- Python (LLM testing harnesses)
- OpenAI / Azure OpenAI APIs
- Git + CI integration
- AWS AgentCore
- AWS CloudWatch / Grafana / Prometheus
- Cursor AI / GitHub Copilot / Claude CLI
Good To Have Tools
- AutoGen
- Promptfoo
- LlamaIndex
- NLP libraries (spaCy, NLTK)
- Agent simulators
- TruLens
- UI Path AI Automation
Nice To Have Tools
- Claude
- Gemini
- Vector DB UIs
- HuggingFace Spaces
- Low‑code GenAI tools
Communication
- Strong verbal and written communication skills, with the ability to clearly articulate defects, risks, and quality insights.
- AI & GenAI Testing Skills:
- Hands‑on experience in testing AI/LLM‑based systems, including prompt‑driven workflows, RAG pipelines, and agent‑based solutions.
- Strong exposure to prompt engineering, prompt validation, and response evaluation techniques.
Additional Information
- Travel Requirements: Travel to client locations may be required as per project requirements.
- Shift Requirements: Required to work on shift as per project requirements.
- Line of Service: Advisory
- Industry: Enterprise Testing Managed Services
- Designation: Senior Associate
- Location: Bangalore, India (ONLY)
Position Level
Senior Associate
Number of Openings
1
Target Location
India AC (Bangalore)
Demand Justification
Specific Client Need
Client Name
Lennar
Client Utilization %
100%
Tower Alignment
CEDA
Target Resource Start Date
03/20/2026
Duration of Engagement
12 months
Tracker ID
Click on Apply to know more.