GiantMind Solutions
Website:
gmindsolution.com
Job details:
Job Title: AI Quality Analyst (French – Personalization Evaluation)Role OverviewWe are seeking a detail-oriented and analytical AI Quality Analyst to evaluate personalization capabilities within advanced AI systems. This role focuses on assessing how effectively the model utilizes user-specific data (such as past interactions, search history, and activity signals) to deliver accurate, relevant, and context-aware responses.
The ideal candidate will combine analytical rigor with creative thinking, designing realistic user prompts and evaluating AI-generated outputs across multiple quality dimensions.
Key Responsibilities- Design and execute multi-turn conversational prompts (1–5 turns) reflecting real-world user scenarios
- Evaluate AI responses for:
- Grounding: Accuracy and factual alignment with user data
- Integration: Natural and seamless use of personalization signals
- Helpfulness: Relevance, clarity, and usability of responses
- Perform side-by-side (SxS) comparisons to determine the most effective response
- Identify and document issues such as:
- Incorrect or unsupported personalization
- Weak or misleading inferences
- Hallucinations or overgeneralizations
- Provide clear, structured, and defensible rationales referencing specific conversation turns
- Validate debug information to ensure proper utilization of data sources
- Maintain data hygiene by removing evaluation conversations to prevent bias
Key Qualifications- French Language Proficiency: Strong reading and writing skills (mandatory)
- Analytical Skills: Ability to assess nuanced and ambiguous outputs
- Prompt Design Skills: Experience in crafting contextual and multi-turn prompts
- Attention to Detail: Ability to identify subtle differences in response quality
- Evaluation Expertise: Understanding of personalization concepts and AI behavior
- Communication Skills: Ability to document findings clearly and concisely
- Work Ethic: Self-motivated and capable of working independently in a remote setup
- Technical Requirements: Laptop/desktop with reliable internet connectivity
Education & Experience- Bachelor’s degree or equivalent in a relevant field (e.g., Linguistics, Journalism, Computer Science, Law, Policy, Ethics, or related disciplines)
- Prior experience in AI evaluation, data annotation, content moderation, or quality analysis is preferred
Engagement Details- Employment Type: Contract
- Work Schedule: 4 to 40 hours per week
- Timezone Requirement: Minimum 4-hour overlap with PST
- Work Mode: Remote
Preferred Candidate Profile- Strong critical thinking and problem-solving abilities
- High level of accuracy and attention to detail
- Comfortable working with evolving guidelines and ambiguous scenarios
- Interest in AI systems, personalization, and user experience
Click on Apply to know more.