Codefeast
Website:
codefeast.in
Job details:
Role Overview:
We are seeking highly qualified Subject Matter Experts (SMEs) across a wide range of academic, research, and industry disciplines to support the evaluation, testing, and improvement of state-of-the-art Large Language Models (LLMs).
In this role, you will leverage your expertise to evaluate AI-generated responses, design domain-specific prompts, identify knowledge gaps, and provide evidence-based feedback to improve the quality and reasoning capabilities of advanced AI systems.
Key Responsibilities
- Create complex, domain-specific prompts to test LLM capabilities.
- Evaluate AI-generated responses for factual accuracy, reasoning, and quality.
- Identify factual errors, logical inconsistencies, and model weaknesses.
- Develop challenging test cases that expose model limitations.
- Provide clear, evidence-based feedback to improve model performance.
Day - to - day task
- Write domain-specific prompts of varying difficulty.
- Compare multiple AI responses and rank them.
- Explain why a response is correct or incorrect using authoritative references.
- Identify ambiguous questions and propose improved prompts.
Minimum Qualifications
- Master's degree or higher in a relevant domain.
- Minimum 3 years of professional, research, or teaching experience.
- Excellent written English communication skills.
- Strong analytical skills and exceptional attention to detail.
Preferred Qualifications
- Experience with LLMs, Generative AI, prompt engineering, or AI evaluation.
- Published research, industry recognition, or teaching experience is a plus.
- Ability to work independently and provide objective, evidence-backed reviews.
Click on Apply to know more.
