About the Role
Nasiko is a control plane for AI agents. In plain terms: people bring their own AI agents, and Nasiko builds them, runs them, connects them to each other, routes work to them, and keeps them healthy at scale.
We are hiring an Agentic QA Engineer to test the platform itself — and to invent how that testing is done. This is not a normal QA job.
Part of Nasiko is ordinary software that behaves the same way every time. But part of it is "smart": it uses AI to decide which agent should handle a request and how several agents should work together. AI doesn't give the same answer twice, so the usual "check that the output matches X" approach breaks. Working out how to test something that changes on every run is the heart of this role.
You will blend three things: classic QA, test automation, and a brand-new kind of testing for AI-driven systems.
What You Test — and What You Don't
This is the most important line in this job:
You test how the platform behaves, including the smart decisions it makes. You do not grade the agents that customers build and run on it.
If a customer's agent gives a bad answer, that's the customer's problem to fix. But Nasiko makes its own decisions — which agent to send a request to, how to run a chain of agents, when to stop an agent that's stuck, how to keep one customer's data away from another's. Those decisions are ours. Testing them is your job.
The Three Kinds of Testing You'll Do
1. Functional testing — does the platform do what it promises? A customer uploads an agent, and Nasiko builds it, deploys it, makes it reachable, and routes requests to it. You test that whole journey, across every agent framework we support, and you test what happens when a step fails.
2. Automation and reliability testing — does it keep working? You turn those tests into automated suites that run on every change, so bugs are caught before release. You also break things on purpose — kill a service, cut a database connection, send a flood of traffic — and check that the platform recovers. You set clear targets (for example, "routing works 99% of the time") and make sure we hit them.
3. Agentic testing — does the platform make good smart decisions? This is the new part. You test the AI-driven behavior of the platform:
- Does it route each request to the right agent? (Today this is not measured at all — you would make it measurable.)
- When several agents work together, does the platform pass information correctly, stay within its cost and step limits, and actually finish?
- If an agent gets stuck in a loop, does the platform stop it?
- If an agent tries to leak private data across customers, does the platform block it?
The trick that makes this possible: you build a set of fake test agents with known, fixed behavior — one that always loops, one that always tries to leak data, one that always overspends, one that always fails a tool call. Real customer agents are unpredictable, so you can't use them to test the platform cleanly. Fake agents with known behavior let you check exactly how the platform reacts.
What You Will Do
- Build the test suite for the platform's smart behavior — routing and multi-agent runs — with clear "good vs. bad" baselines.
- Set reliability targets (SLOs) for these behaviors, and add a release gate that blocks a deploy when quality drops.
- Build and maintain the fleet of fake test agents described above.
- Break things on purpose: make tools fail, make agents hang, run out of budget mid-task — and confirm the platform handles it gracefully.
- Build a "red team" that throws tricky and hostile inputs at the platform to find weak spots.
- Turn the traces we already collect into health metrics and alerts: task success rate, stuck-loop count, routing accuracy, cost per task.
- Write up the failure patterns you find, and clear playbooks for handling each one.
- Own the QA process for new releases, so the platform behaves predictably as we grow.
- Later: help turn these internal tools into a product, so customers can test their own agents on Nasiko.
What We Are Looking For
- A solid background in QA, SDET, or Site Reliability Engineering — you know what good quality engineering looks like.
- Real interest in AI agents and how they behave (and misbehave).
- Experience building automated tests that run in CI/CD.
- Comfortable with AI/ML ideas — you don't need to train models, just understand how they act.
- An adversarial mind — you enjoy asking "how could this break?"
- Strong scripting skills (Python preferred) and comfort with cloud-native tools.
- Clear communication — you'll explain quality issues to engineering, product, and leadership.
Bonus Points
- Prior experience working with LLM-based systems, AI agents, or autonomous pipelines.
- Familiarity with agent frameworks such as LangGraph, AutoGen, CrewAI, or similar.
- Understanding of control plane architecture and distributed systems reliability.
- Experience with observability tools (OpenTelemetry, Datadog, Grafana).
- Contributions to open-source AI or QA tooling.
Why This Role Is Unique
The 'Agentic QA Engineer' does not yet exist as a standardized role in the industry. You will be defining what this function looks like — setting best practices, inventing new testing paradigms, and shaping the reliability culture of an AI-native company. If you are someone who loves being at the frontier and building things that don't exist yet, this is your role.
What We Offer
- A chance to define a brand-new engineering discipline from the ground up.
- Collaborative, fast-moving team that values ownership and autonomy.
- Competitive compensation and benefits.
- Flexible, remote-friendly work environment.
- Direct impact on the reliability and quality of cutting-edge AI products.
Nasiko — Building the Future of Agentic AI
