YO HR Consultancy
Website:
yohrconsultancy.com
Company:
https://www.linkedin.com/company/yo-hr-consultancy-pvt-ltd
Industries: Artificial Intelligence
Job details:
AI Data Quality Lead – Robotics & Video Annotation | Remote
About The Role
We are looking for an experienced
AI Data Quality Lead to own the quality standards for egocentric training data used to develop and evaluate physical-AI and robotics models.
You will be responsible for building and maintaining the
golden sets, annotation SOPs, rubrics, calibration systems, quality frameworks, and audit processes that ensure high-quality ground-truth data. You will also work closely with engineering and product teams to develop annotation workflows for
hand tracking, object identification, and action labeling.
This is a ground-floor opportunity to build a
0→1 data quality system at a fast-growing, well-funded startup. You will not simply inherit an existing process—you will help establish the quality standards, workflows, and systems from the ground up.
Key Responsibilities
- Own the end-to-end quality framework for egocentric training data, including:
- Golden and benchmark datasets
- Acceptance criteria
- Frame-level tolerance thresholds
- Sampling strategies
- Audit and quality-control loops
- Author, maintain, and version annotation SOPs, rubrics, and edge-case guidelines as project specifications evolve.
- Translate ambiguous requirements into precise, actionable instructions for annotators and reviewers.
- Establish and manage annotator training, certification, and calibration programs.
- Run calibration sessions and adjudicate disagreements between annotators.
- Measure and report quality using metrics such as:
- Inter-Annotator Agreement (IAA)
- Kappa/alpha
- Tolerance-based agreement for temporal boundaries
- Acceptance/rejection rates
- Rework rates
- Cost per accepted hour
- Identify root causes of quality issues and determine whether they result from guideline ambiguity, annotation errors, or workflow issues.
- Design human-in-the-loop workflows where AI/model-generated labels are reviewed, corrected, and escalated by human annotators.
- Audit auto-generated labels and improve annotation throughput without compromising quality.
- Partner with engineering and product teams to develop new annotation workflows for:
- Hand tracking
- Object identification
- Action labeling
- Define label schemas, annotation-tool requirements, QC dashboards, and pilot-validation processes.
- Validate annotation workflows through pilot programs before scaling them into production.
- Work with ML teams to understand how annotation quality impacts model performance.
- Contribute to tooling and workflow improvements where required.
Required Skills & Qualifications - 5+ years of hands-on video annotation experience, including at least 1+ year working with egocentric/first-person video for robotics or embodied AI.
- 3+ years of experience in a lead or QA capacity, including annotator calibration, disagreement adjudication, and ownership of annotation guidelines.
- Demonstrated experience building 0→1 annotation programs and taking them from pilot through production.
- Strong command of annotation-quality methodologies, including:
- IAA frameworks
- Kappa/alpha
- Gold-set creation
- Sampling strategies
- Tolerance thresholds
- Acceptance criteria
- Ability to design quality systems and explain the practical difference between quality targets such as 95% and 99% accuracy.
- Proven experience authoring, rather than simply following, annotation SOPs and rubrics.
- Ability to translate ambiguous specifications into clear and actionable annotation guidelines.
- Hands-on experience with at least 2 professional annotation platforms, such as:
- CVAT
- Labelbox
- Encord
- V7
- Label Studio
- Equivalent annotation tools
- Experience configuring label schemas and QA workflows within annotation platforms.
- Experience auditing model-generated or automatically generated labels.
- Experience designing human-in-the-loop annotation workflows.
- Strong written and verbal English communication skills.
- Strong analytical skills and proficiency with Excel and/or Google Sheets for quality reporting.
- Strong understanding of ML concepts and how annotation quality affects model performance.
- Data-driven and process-oriented mindset with strong ownership.
- Comfortable working in ambiguous, fast-moving, startup environments.
- Strong ability to work collaboratively with cross-functional and remote teams.
Preferred / Bonus Qualifications - Background in robotics, mechanical engineering, mechatronics, or computer vision engineering.
- Experience working at a leading robotics company, AI/robotics startup, or similar project.
- Experience managing distributed or global annotator workforces or external annotation vendors.
- Experience with multimodal annotation, including:
- 3D/depth data
- Joint pose
- Grasp outcome classification
- Experience training or fine-tuning autolabeling models.
- Experience partnering closely with ML teams responsible for automated labeling.
- Experience building annotation tooling or working closely with annotation-tooling engineering teams.
- Experience with physical AI, embodied AI, computer vision, or robotics datasets.
Why This Role?
The quality bar is the product.
The ground-truth data produced through this role will become a benchmark against which physical-AI models are measured. You will have direct ownership of the systems that determine whether that data meets the required quality standards.
This is a
ground-floor opportunity at a fast-growing, well-funded startup where you will build the quality system from scratch rather than inherit an established process.
Role Highlights
- Role: AI Data Quality Lead – Robotics & Video Annotation
- Work Type: Remote
- Engagement: [Contract / Full-Time – specify as applicable]
- Focus: AI Training Data, Robotics, Embodied AI & Video Annotation
- Experience: 5+ years relevant experience
Skills: calibration,video,data quality,training,robotics,annotation
Click on Apply to know more.
