AI Data Scientist, Evaluation & Insights

Ironclad is the leading AI-powered contract lifecycle management platform, processing billions of contracts every year. Every business is powered by contracts, but managing them can slow companies down and cost millions of dollars. Global innovators like L'Oréal, OpenAI, and Salesforce trust Ironclad to transform contracting into a strategic advantage - accelerating revenue, reducing risk, and driving efficiency. It's the only platform that manages every type of contract workflow, whether a sales agreement, an HR agreement or a complex NDA. We're building the future of intelligent contracting and writing the narrative for how contracts unlock strategic growth. Forrester Wave and Gartner Magic Quadrant have consistently recognized Ironclad as a leader in our category. We've also been named one of Fortune's Great Places to Work six years running, featured on Glassdoor's Best Places to Work, and recognized by Forbes' 50 Most Promising AI Companies. We're backed by leading investors like Accel, Sequoia, Y Combinator, and BOND. We'd love for you to join us! About the Role Ironclad is accelerating its investment in AI to redefine how legal teams manage and understand contracts. As part of this effort, we are hiring an AI Evaluation Engineer to work within our AI Pillar. This role is focused on unlocking insights from our training data, designing feedback loops, and ensuring the continuous improvement of our agentic and ML or LLM-based systems through data-driven evaluation and iteration. You'll partner closely with AI Engineers and Product Managers to drive better model quality through systematic analysis, experimentation, and the curation of high-leverage datasets. Your work will directly impact the effectiveness of features like Smart Import, contract understanding, and agentic workflows. What You'll Be Doing Analyze training and evaluation datasets to identify distributional gaps, labeling inconsistencies, and long-tail opportunities. Design and execute labeling campaigns, including development of golden datasets and annotation guidelines. Build and maintain dashboards that track model accuracy, regression trends, and product-specific KPIs like success rate or answer helpfulness. Investigate failure modes via prompt clustering, error taxonomy development, and user intent classification. Operationalize feedback loops: mine product telemetry and human-in-the-loop reviews for signal, then translate into data-driven model improvement strategies. Partner with engineers and PMs to run structured A/B tests and human evaluations for new models or features. Support the development of scalable data and evaluation infrastructure for LLMs and agents. Work with product, engineering and legal to create clear & transparent processes for the handling of customer data in AI training, fine-tuning and evaluation About You Bachelor's or Master's degree in a quantitative field (e.g., Statistics, Computer Science, Data Science, Applied Math). 1–3 years of experience in applied ML or data science, preferably in NLP or LLM-based applications. Strong SQL and Python skills; experience with Jupyter, Pandas, and experiment tracking tools. Comfortable navigating ambiguity, slicing large datasets, and communicating insights clearly to cross-functional stakeholders. Experience with prompt analysis, clustering, or user behavior modeling is a plus. Bonus: familiarity with LLM eval techniques, Reinforcement Learning from Human Feedback (RLHF), or agentic system design.Experience with program management.

Ironclad is the #1 contract lifecycle management platform for innovative companies. Every company, in every country, in every industry runs on contracts, but managing these contracts slows companies down and costs them millions of dollars. L'Oréal, Staples, Mastercard, Dropbox, DoorDash, Snap Inc., and other leading innovators use Ironclad to collaborate and negotiate on contracts, accelerate contracting while maintaining compliance, and turn contracts into critical carriers of operational business intelligence. It's the only platform flexible enough to handle every type of contract workflow, whether a sales agreement, an HR agreement, or a complex NDA. Ironclad recently reached a $3.2B valuation with a $150M Series E Investment round. Ironclad's innovation and work culture has been recognized by Forbes' 50 Most Promising AI Companies in 2021, Fortune's Best Small & Medium Workplaces in 2021, Battery Ventures and Glassdoor's 25 Highest-Rated Cloud Computing Company Employers, Fast Company's 2020 Best Workplaces for Innovators, Glassdoor's Best Machine Learning Startups, and Wing Venture Capital's Enterprise Tech 30 List. Ironclad is backed by leading investors like Accel, Y Combinator, Sequoia, and BOND.