Crossover
Website:
crossover.com
Job details:
If accuracy matters more to you than speed, this position is built for that mindset. The labels you create become training data for AI systems used daily by thousands of students. Precise behavioral annotation improves the model. Inconsistent labeling teaches it the wrong patterns.
LearnWith.AI develops AI-driven learning experiences grounded in learning science, data analytics, and expert knowledge. This position transforms raw student session recordings into high-fidelity, rubric-aligned labels the team can depend on. You will review recorded student sessions, pinpoint key behavioral moments, and follow strict classification protocols to mark what occurred and exactly when. You will also assess LLM-generated pre-annotations, correct errors, and flag edge cases to help engineers refine the system.
This is not fragmented, gig-based annotation work. It is a focused queue within a single product area, supported by direct feedback channels, gold-standard calibration, and advancement tied to accuracy and consistency. If you value explicit standards, quantifiable quality, and contributions that directly influence model outcomes, we would like to hear from you.
What You Will Be Doing
- Review student session video recordings to identify, classify, and timestamp behavioral events according to a comprehensive rubric
- Evaluate and refine LLM-generated pre-annotations by eliminating false positives, inserting overlooked events, and adjusting timestamps
- Document reasoning for ambiguous decisions with clear notes, including rubric citations and the logic applied
- Record edge cases and submit clarification requests for unclear scenarios while maintaining an updated annotation tracker with session details
- Participate in calibration sessions, integrate QA feedback, and apply rubric revisions to enhance accuracy continuously
What You Won’t Be Doing
- Develop AI models, conduct experiments, or perform research on student behavioral patterns
- Create or modify the annotation rubric or alter category definitions based on subjective judgment
- Prioritize annotation speed over accuracy, consistency, or precise timestamping
- Handle scattered, one-time tasks across disconnected domains without context or feedback mechanisms
Annotation Specialist Key Responsibilities
This position ensures that student session videos are transformed into labeled datasets with ≥95% accuracy and time-precise annotations that dependably indicate whether model performance is advancing or declining.
Basic Requirements
- A minimum of 1 year of experience in data annotation, content moderation, QA evaluation, or comparable rubric-based review roles
- Excellent English reading comprehension with the ability to adhere to detailed written instructions without deviation
- Capacity to maintain focus and precision during 4–6 hours of video review work each day
- Skill in detecting subtle visual and on-screen behavioral signals and categorizing them uniformly across multiple sessions
- Proficient written documentation abilities for clarifying edge cases, reasoning, and questions
- Dependable internet connection sufficient for video streaming
- Experience reviewing, correcting, and enhancing AI/LLM-produced annotations
About LearnWith.AI
LearnWith.AI is an edtech startup that leverages AI and subject matter experts to cultivate a new way of learning. Our unique approach leverages 50+ years of learning science, cutting-edge data analytics and AI-powered coaching. In doing so, we can help students learn more, learn faster, and learn better - and have fun while doing it. We are a remote-first company that hires globally via Crossover.
There is so much to cover for this exciting role, and space here is limited. Hit the Apply button if you found this interesting and want to learn more. We look forward to meeting you!
Working with us
This is a full-time (40 hours per week), long-term position. The position is immediately available and requires entering into an independent contractor agreement with Crossover as a Contractor of Record. The compensation level for this role is $15 USD/hour, which equates to $30,000 USD/year assuming 40 hours per week and 50 weeks per year. The payment period is weekly. Consult www.crossover.com/help-and-faqs for more details on this topic.
Crossover Job Code: LJ-5785-IN-Patna-AnnotationSpec.002
Click on Apply to know more.