Cerebry
Website:
cerebry.co
Job details:
About the Role
We are seeking an experienced Mathematics Assessment Specialist to review, refine, and finalize AI-generated mathematics questions designed for curriculum-aligned, international-standard assessments.
Our assessments emphasize mathematical literacy, conceptual understanding, reasoning, problem-solving, and the application of mathematics to real-world situations.
This is a hands-on assessment development and quality assurance role. You will work with AI-powered question generation and evaluation systems, applying your expertise to ensure that every question is mathematically accurate, educationally meaningful, appropriately calibrated, and exam-ready.
We are looking for professionals with strong assessment experience, sound analytical judgment, and the ability to independently evaluate both questions and AI-generated recommendations.
Key Responsibilities
1. Question Review and Improvement
- Critically review AI-generated mathematics questions for mathematical correctness, clarity, relevance, and assessment quality.
- Identify weaknesses in question design, reasoning requirements, real-world contexts, difficulty levels, and answer options.
- Directly edit and rewrite questions, solutions, explanations, and distractors to make them exam-ready.
- Ensure questions have unambiguous wording, valid answers, and appropriate difficulty levels.
- Evaluate whether questions genuinely assess mathematical thinking rather than merely procedural knowledge.
2. AI-Assisted Assessment Development
- Use AI tools and effective prompting techniques to generate, refine, and improve assessment questions.
- Evaluate AI-generated feedback and recommendations critically.
- Distinguish between valid feedback, unnecessary suggestions, and incorrect AI assessments.
- Accept, reject, or modify AI recommendations based on sound mathematical and assessment principles.
- Identify cases where AI has introduced errors, misunderstood a question, or suggested changes that reduce assessment quality.
- Iterate with AI tools and make direct editorial changes as needed to produce final, exam-ready questions.
3. International Assessment Quality
- Ensure alignment with international assessment principles, particularly those reflected in PISA, TIMSS, and comparable programmes.
- Evaluate the authenticity and educational relevance of real-life situations used in questions.
- Assess cognitive demand, mathematical literacy, reasoning, and problem-solving requirements.
- Ensure questions are appropriate for the intended student age group and proficiency level.
- Maintain consistent quality standards across questions and assessments.
4. Quality Assurance and Finalization
- Take ownership of the final quality of assigned assessment items.
- Verify mathematical solutions, scoring guidelines, and answer keys.
- Ensure that diagrams, graphs, tables, numerical information, and contexts are accurate and consistent.
- Document significant changes and provide concise explanations for important editorial decisions.
- Flag questions that cannot be satisfactorily improved and recommend replacement where appropriate.
Required Qualifications and Experience
- Minimum 5 years of relevant professional experience in mathematics assessment development, item writing, item review, or assessment quality assurance.
- Demonstrated experience developing or reviewing competency-based, application-oriented, or real-world mathematics questions.
- Strong expertise in mathematical reasoning, problem-solving, assessment design, and question validation.
- Ability to independently identify conceptual, mathematical, linguistic, and assessment-design weaknesses.
- Excellent analytical skills, critical thinking, and attention to detail.
- Strong written communication skills and the ability to explain assessment decisions clearly.
- Comfortable working with digital tools and willing to use AI-based systems for assessment development.
Prior professional experience with AI tools is beneficial but not mandatory. Strong assessment expertise and independent judgment are more important.
Highly Preferred
- Direct experience with PISA, TIMSS, Cambridge International, IB, or comparable international assessment programmes.
- Experience with large-scale assessments, standardized testing, or international assessment organizations.
- Familiarity with assessment frameworks, cognitive demand, item difficulty, scoring rubrics, and psychometric principles.
- Experience developing questions involving mathematical modelling, data interpretation, quantitative reasoning, and authentic real-world scenarios.
What Makes a Strong Candidate?
We particularly value professionals who:
- Can recognize when a question is technically correct but still poorly designed.
- Can transform an average question into an excellent assessment item.
- Can independently challenge AI-generated evaluations rather than accepting them at face value.
- Know when AI feedback is useful, unnecessary, or incorrect.
- Can balance mathematical accuracy, assessment validity, clarity, and real-world relevance.
- Apply logical reasoning and sound professional judgment to ambiguous or complex situations.
- Take ownership of delivering polished, exam-ready questions rather than simply reporting problems.
Important Eligibility Requirement
This position is intended for experienced assessment professionals.
General mathematics teaching experience or subject-matter expertise alone, without demonstrable assessment development or review experience, is insufficient.
Selection Process
Shortlisted applicants will complete a practical assessment involving AI-generated mathematics questions.
Candidates will be expected to:
- Identify mathematical, conceptual, and assessment-design weaknesses.
- Critically evaluate AI-generated feedback, including identifying incorrect or unnecessary recommendations.
- Improve questions through direct editing or AI-assisted prompting.
- Produce final, exam-ready assessment items.
- Briefly justify significant decisions and changes.
Please include details of your previous assessment-development projects, international assessment experience, and specific responsibilities when applying.
Click on Apply to know more.