Website:
fetchjobs.co
Job details:
About The Company
Based in San Francisco, California, Turing is the world’s leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. Turing supports customers in two ways: first, by accelerating frontier research with high-quality data, advanced training pipelines, plus top AI researchers who specialize in coding, reasoning, STEM, multilinguality, multimodality, and agents; and second, by applying that expertise to help enterprises transform AI from proof of concept into proprietary intelligence with systems that perform reliably, deliver measurable impact, and drive lasting results on the P&L.
About The Role
As an AI Quality Analyst at Turing, you will play a vital role in evaluating and enhancing the personalization features of our cutting-edge AI models, specifically for the Gemini project. Your primary responsibility will be to assess how effectively the AI utilizes information from various personal data sources, including past conversations, Gmail, Google Search, and YouTube activity, to generate more relevant and helpful responses. This role demands a combination of analytical precision and creative thinking. You will design prompts based on your personal experiences and evaluate the AI’s responses, focusing on dimensions such as Grounding, Integration, and Helpfulness. Your insights will directly influence the development of more personalized, reliable, and user-centric AI systems.
Qualifications
- Proficiency in Hindi: Ability to read and write fluently in Hindi, as it is the primary language for this project.
- Personal Account Usage: Willingness to use your personal Google account (not a test or secondary account) and enable personal data sources for genuine assessment purposes.
- Schedule Flexibility: Full-time availability within your local time zone, with at least 4 hours of overlap with Pacific Standard Time (PST).
- Analytical Skills: Demonstrated ability to evaluate nuanced AI responses, especially in the context of personalization quality.
- Creative Prompt Engineering: Experience in designing multi-turn prompts that incorporate personal context to thoroughly test AI capabilities.
- Evaluation Expertise: Strong understanding of personalization concepts, including identifying incorrect inferences, poor personalization, and forced connections.
- Attention to Detail: Ability to meticulously review side-by-side model responses, noting subtle differences in naturalness and coherence.
- Communication Skills: Excellent written skills to provide clear, structured rationales referencing specific conversation turns.
- Constructive Feedback: Ability to give detailed annotations and actionable insights to improve AI responses.
- Independence: Self-motivated with the ability to work autonomously in a remote environment.
- Technical Setup: Reliable desktop or laptop with a stable internet connection.
Responsibilities
- Design and execute multi-turn conversational prompts (typically 1-5 turns) that require the AI to utilize your personal information and experiences effectively.
- Assess whether the AI’s responses appropriately incorporate personalization based on your prompts and data sources.
- Identify and address Grounding issues, ensuring claims about your personal data are supported by evidence and free from hallucinations or inaccuracies.
- Evaluate the quality of data integration, ensuring personal data is woven into responses naturally without robotic overnarration.
- Compare and rank two model responses side-by-side, determining which is more helpful, engaging, and easy to use.
- Provide detailed rationales for your rankings, explicitly referencing specific turns and aspects of the conversation.
- Verify that chat summaries and data sources are properly utilized by extracting and reviewing debug information from the model.
- Maintain strict data hygiene by deleting evaluation conversations after review to prevent contamination of future chat history.
- Collaborate with team members to share insights and improve evaluation methodologies.
Benefits
This role offers an excellent opportunity to be part of innovative AI research and development. You will work remotely with flexible hours, allowing you to balance your professional and personal commitments. The engagement provides a competitive hourly rate of $15, with a commitment of at least 30 hours per week, ensuring a steady income over the three-month contract period. You will gain valuable experience in AI evaluation, personalization, and data privacy practices, working alongside some of the best minds in the field. Additionally, this position offers the chance to contribute directly to the development of next-generation AI systems that impact millions of users worldwide.
Equal Opportunity
Turing is committed to creating an inclusive environment for all employees and applicants. We are an equal opportunity employer and do not discriminate based on race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status. We believe in fostering a diverse workplace that reflects the global community we serve, and we welcome applications from individuals of all backgrounds and experiences.
Click on Apply to know more.