Urdu LLM Evaluator

Join Project Spearmint, a multilingual AI response evaluation project reviewing large language model (LLM) outputs in different languages, focused on either Tone or Fluency. Native-level fluency in a target language, along with strong English comprehension, is required. As an evaluator, you will review short, pre-segmented datasets and assess model-generated replies based on specific quality dimensions. Your input will help validate evaluation frameworks and establish baseline quality metrics for future model development. Key Responsibilities: - Evaluate model replies in your native language based on either Tone or Fluency. - Assess the overall quality, correctness, and naturalness of responses. - Read the user prompt and two model replies, then rate each using a five-point scale. - Provide brief rationales for any extreme ratings. Project Breakdown: Batch 1 – Tone: Determine whether replies are helpful, insightful, engaging, and fair. Flag formality mismatches, condescension, bias

CrowdGen by Appen