Website:
anthrobyte.ai
Job details:
Anthrobyte wins by putting AI agents into the real operations of enterprise clients — and making them work, in production, where it matters. Production is only trusted when it has been proven. We need an engineer who can prove it: someone who tests AI systems the way they actually behave, not the way a demo suggests, and who builds the tooling that lets the whole team do the same.
This is a build-from-the-front role. You test inside the problem, against the client’s real data and constraints, not from a checklist.
What you’ll own
- Quality for AI in production. You decide what “good enough to ship” means for systems that don’t behave the same way twice. Evaluation sets, regression suites, hallucination and guardrail checks — you design them, run them, and defend the results.
- Performance and benchmarking. You build repeatable benchmarks for latency, throughput, and retrieval quality across pipelines, vector search, and agent workflows — and you explain why the numbers moved.
- Quality gates in the pipeline. You embed automated tests into CI/CD so nothing reaches a client’s live operations without passing them. The gate is yours, and so is the call when it should stay shut.
- The product loop. You turn what you learn in the field into reusable test frameworks, benchmarks, and standards inside AATLAS — so every engagement makes the next one faster and safer, and we compound into a product, not a services treadmill.
- Raising the standard. This is the multiplier. You mentor engineers in test automation and performance practice, and codify how we test into a playbook others can run without you. Success is a team that tests the way you do, not you testing everything.
- Client trust. You present benchmark and quality findings to engineering leads, client security, and business stakeholders — in architecture reviews, security reviews, and the room where decisions get made — and you hold your position when the answer is “not ready yet.”
Requirements
What we’re looking for
- Production track record. 7–10 years in software testing, at least 5 of them as an SDET or automation engineer. You’ve built test frameworks from scratch and lived with what you shipped.
- Core automation, with depth. Expert in Python and Pytest — framework design, fixtures, parametrization, reporting — and in Postman, with API suites automated in CI. Production experience in at least one UI framework: Playwright, Cypress, or Selenium.
- Performance engineering. Hands-on with Locust, Gatling, k6, or JMeter, and able to interpret results rather than just generate them.
- Delivery plumbing. CI/CD in GitLab, GitHub Actions, or Azure DevOps. Docker. At least one cloud platform (AWS or Azure). Backend and database testing.
- Range. You’re equally comfortable writing code, designing a test strategy, and explaining a failure to a non-technical stakeholder.
- Systems mindset. You think in reliability, not test counts. Enterprise AI is won on auditability and things that keep working.Bonus points
- Experience testing ML and LLM systems, RAG pipelines, vector databases, embeddings, or MCP-based agent workflows.
- Familiarity with LLM evaluation approaches and tooling.
- A second UI framework, Jest, or BDD tooling (SpecFlow, pytest-bdd, behave).
- Accessibility and security testing (axe, OWASP ZAP).
- Experience in regulated or enterprise environments such as finance, healthcare, or telecom.
- A track record of growing junior engineers into independent operators.
What this role is not
A manual tester working from scripts. A pure automation engineer who runs suites someone else designed. A researcher who benchmarks in isolation and never sees a client’s reality. If you want to test from a distance rather than from inside the problem, this isn’t it.
How we work
Small, high-trust, founder-close team. Governance-first, because our work runs inside clients’ live operations. You’ll have real authority over what “ready” means, and the mandate to build the testing capability we’ll become known for.
How to apply
Tell us two stories.
First: a system you tested or benchmarked where the numbers surprised you, what you built to find out, and what you did when the answer was uncomfortable.
Second: a release you heldback, or a standard you set, that others pushed against, and how it played out. Those two stories tell us more than any CV.
Click on Apply to know more.
