GiantMind Solutions
Website:
gmindsolution.com
Job details:
Skill: Experience in data annotation, AI quality evaluation, content moderation, or a related role is strongly preferred.
Availability: Full Time Availability (8 Hrs) and 4 hours overlap with PST time zone.
Education & Experience: BS/BA degree or equivalent experience in Policy, Law, Ethics, Linguistics, Journalism, Computer Science, or related analytical field.
Key Responsibilities- High-level language proficiency (reading, writing, comprehension) in assigned language.
- Willingness to use primary personal Google account (for applicable roles).
- Full-time schedule flexibility supporting global 24-hour operations.
- Evaluate nuanced and ambiguous AI responses focusing on personalization quality.
- Design creative multi-turn starting prompts using personal context.
- Identify incorrect personalization, poor inferences, hallucinations, and grounding issues.
- Review Side-by-Side (SxS) model responses and detect subtle differences in naturalness.
- Write clear, structured rationales referencing specific conversation turns.
- Provide detailed annotations, feedback, and collaboration in a remote setup.
- Maintain proper technical setup (Desktop/Laptop with stable internet).
Description- Design and execute multi-turn conversational prompts (1–5 turns).
- Evaluate AI responses based on personalization intent and user context.
- Analyze responses for grounding issues and unsupported assumptions.
- Assess integration quality ensuring personal data is used naturally.
- Perform rigorous SxS model comparison and ranking.
- Write clear and defensible evaluation rationales.
- Extract and verify debug information from model outputs.
- Maintain strict data hygiene by deleting evaluation conversations.
Click on Apply to know more.