About the role
๐ ๐๐ ๐ฅ๐ฒ๐๐ฒ๐ฎ๐ฟ๐ฐ๐ต ๐๐ป๐ด๐ถ๐ป๐ฒ๐ฒ๐ฟ (3+ ๐ฌ๐ฒ๐ฎ๐ฟ๐ ๐ผ๐ณ ๐๐
๐ฝ๐ฒ๐ฟ๐ถ๐ฒ๐ป๐ฐ๐ฒ)
Weโre seeking an AI Research Engineer to own model evaluations end-to-end, turning ambiguous capabilities into rigorous, reproducible benchmarks. Working closely with researchers, you will design datasets, analyze model behavior at scale, and directly shape the training lifecycle.
๐งฉ ๐ช๐ต๐ฎ๐ ๐ฌ๐ผ๐'๐น๐น ๐๐ผ
โข Design rigorous benchmarks for reasoning, coding, and agentic capabilities across C/C++, Rust, embedded systems, hardware specifications, and engineering tasks.
โข Develop deterministic evaluators using compilers, emulators, simulators, static analysis, formal verification, and hardware-in-the-loop systems.
โข Build distributed infrastructure to run evaluations reliably against models and live training checkpoints.
โข Diagnose regressions and anomalous results, separating model failures from issues in prompts, data, evaluators, or infrastructure.
โข Design robust metrics, difficulty curricula, contamination-resistant tasks, and experiments around prompting, sampling, and scaffolding.
โข Turn evaluation failures into targeted datasets, reward signals, and new training objectives in collaboration with RL researchers.
โข Create dashboards, experiment tracking, and evaluation libraries that make model performance easy to understand and trust.
โ
๐ฌ๐ผ๐'๐ฟ๐ฒ ๐ฎ ๐๐ฟ๐ฒ๐ฎ๐ ๐๐ถ๐ ๐๐ณ ๐ฌ๐ผ๐
โข Strong Python programming and software engineering fundamentals.
โข Experience building benchmarks, evaluation systems, automated graders, research infrastructure, or similar testbeds.
โข Strong understanding of LLMs and modern evaluation methodologies.
โข Strong analytical and experimental thinking; you care deeply about whether a metric actually measures the intended capability.
โข Experience debugging complex systems and investigating unexpected experimental results.
โข Strong written and verbal communication.
โจ ๐ช๐ต๐ ๐๐ผ๐ถ๐ป ๐จ๐?
โข Own the development of cutting-edge infrastructure from day one.
โข Solve complex engineering challenges that directly shape our product roadmap.
โข Accelerate your career alongside elite researchers in a fast-paced environment.
โข Work closely with a brilliant, collaborative team of industry pioneers.
This page is fully interactive when JavaScript is enabled. Please enable JavaScript to apply or browse related roles.