AI Quality Engineer

Website: fetchjobs.co
Job details:
About The Company

Based in San Francisco, California, Turing is the world’s leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. Turing supports its clients by accelerating frontier research through high-quality data, sophisticated training pipelines, and top-tier AI researchers specializing in coding, reasoning, STEM, multilinguality, multimodality, and agents. Additionally, Turing helps enterprises transform AI from proof of concept into proprietary intelligence by developing reliable systems that deliver measurable impact, enhance decision-making, and drive sustainable growth. The company's mission is to push the boundaries of AI innovation while ensuring responsible and ethical deployment across various industries.

About The Role

As an AI Quality Analyst at Turing, you will play a critical role in evaluating and enhancing the personalization features of Gemini, an advanced AI system. Your primary responsibility will be to assess how effectively the model leverages personal information from conversations, Gmail, Google Search, and YouTube activity to generate relevant and helpful responses. This role demands a combination of creativity and analytical rigor, as you will design prompts based on your personal experiences and evaluate the model’s responses for quality, grounding, integration, and helpfulness. Your insights will directly influence the refinement of AI personalization capabilities, ensuring that the system provides accurate, contextually appropriate, and user-centric interactions.

Qualifications

• Proficiency in reading and writing in Hindi with a high degree of competency, as Hindi is the focus language for this project.
• Willingness to use your primary personal Google account and enable personal data sources for authentic assessment purposes.
• Full-time availability in your local time zone, with flexibility to accommodate a global, 24-hour operations schedule.
• Exceptional analytical thinking skills, especially in evaluating nuanced AI responses and personalization quality.
• Experience in designing creative, multi-turn prompts based on personal context to thoroughly test AI capabilities.
• Strong evaluation skills to identify incorrect personalization, poor inferences, and forced connections.
• Meticulous attention to detail to review Side-by-Side (SxS) model responses and detect subtle differences in naturalness and overnarration.
• Excellent written communication skills to articulate clear, concise rationales for model rankings, referencing specific conversation turns.
• Ability to provide constructive feedback and detailed annotations to support continuous improvement.
• Strong collaboration and communication skills for effective remote teamwork.
• Self-motivated and capable of working independently in a remote setup.
• Reliable technical setup with a desktop or laptop and a stable internet connection.
  
  

Responsibilities

In this role, you will be part of a dynamic team focused on evaluating the quality of personalized AI interactions. Your daily tasks will include designing multi-turn conversational prompts (typically 1-5 turns) that require the AI to utilize your personal information and experiences effectively. You will analyze model responses to ensure personalization is appropriately applied, checking for grounding issues where claims about you are supported by evidence and not flawed inferences or hallucinations. Your evaluations will also assess the natural integration of personal data into responses, avoiding robotic overnarration.

Further, you will rigorously compare two model responses side-by-side, determining which one is more helpful, natural, and user-friendly. You will write clear rationales for your comparisons, explicitly referencing specific conversation turns to justify your assessments. Extracting and verifying debug information from the model will be essential to confirm that chat summaries and data sources are properly utilized. Maintaining strict data hygiene by deleting evaluation conversations is crucial to prevent contamination of your future chat history.

Benefits

As a contractor with Turing, you will enjoy flexible working hours, allowing you to balance your professional and personal commitments. You will have the opportunity to work remotely with a global team, gaining exposure to cutting-edge AI research and development projects. The engagement offers a competitive hourly rate of $15, with options for 30 or 40 hours per week, based on your availability. This role provides valuable experience in AI evaluation, personalization, and data annotation, which can enhance your professional profile. Additionally, Turing fosters a collaborative and inclusive work environment, supporting continuous learning and growth through feedback and engagement with top AI experts.

Equal Opportunity

Turing is committed to creating an inclusive environment and is proud to be an equal opportunity employer. We do not discriminate based on race, religion, gender, sexual orientation, age, disability, or any other protected characteristic. All qualified applicants will receive consideration for employment without regard to these factors. We believe diversity and inclusion are essential to fostering innovation and achieving our mission of advancing AI technology responsibly and ethically. Click on Apply to know more.