Teleradiology Solutions
Website:
telradsol.com
Company:
https://www.linkedin.com/company/teleradiology-solutions
Seniority: Not Applicable
Industries: Hospitals and Health Care
Job details:
Job Title: Testing Lead – Deep Learning / LLM / VLM
Location
Remote / Hybrid / On-site
Experience
3–4 years (hands-on ownership in DL / LLM / GenAI testing)
Employment Type
Full-time
Role Overview
We are seeking a
hands-on Testing Lead to own quality and
documentation for our Deep Learning, LLM, and Vision-Language Model (VLM) products. You will define how we test, measure, document, and communicate AI quality—working closely with ML, Engineering, and Product teams in a fast-paced startup environment.
This role is ideal for someone who believes
clear documentation is as critical as good testing, especially for non-deterministic AI systems.
What You’ll Do
Own Quality & Documentation End-to-End
- Define testing strategy for LLMs, VLMs, and DL pipelines.
- Create and maintain clear, lightweight documentation covering:
- Model testing strategies and assumptions
- Evaluation metrics and acceptance criteria
- Known limitations, risks, and failure modes
- Release readiness and quality sign-off
- Ensure documentation evolves with models, data, and prompts.
LLM / GenAI Testing
- Design tests for:
- Prompt templates and prompt changes
- RAG pipelines (retrieval quality, grounding, hallucination control)
- Multi-turn conversations and long-context behaviour
- Maintain golden datasets, regression test suites, and test result summaries.
- Document prompt behaviour, edge cases, and known model quirks.
Vision & Multimodal Testing
- Test VLMs for image-text alignment, OCR, captioning, and reasoning.
- Document model performance across different image types, quality levels, and domains.
- Track and publish model behaviour changes between versions.
Automation, MLOps & Reporting
- Build Python-based automation for evaluation and regression testing.
- Integrate tests into CI/CD and MLOps pipelines.
- Produce readable quality reports and dashboards for engineers and leadership.
- Monitor and document production issues such as model/data drift and degradation.
Build a Quality-First Culture
- Establish QA and documentation standards that scale with a startup.
- Mentor engineers on writing testable code and meaningful documentation.
- Act as the single source of truth for AI quality, testing, and known risks.
What We’re Looking For
Must-Have
- Strong background in software testing with lead or ownership experience.
- Hands-on experience testing LLMs, DL models, or GenAI systems.
- Strong Python skills for test automation and data validation.
- Proven ability to write clear, structured technical documentation.
- Understanding of:
- Transformer-based models and DL workflows
- Model evaluation metrics and non-deterministic system testing
- Comfortable working in ambiguity and moving fast in a startup.
Nice-to-Have
- Experience with VLMs, multimodal models, or computer vision.
- Exposure to RAG architectures, vector databases, and embeddings.
- Familiarity with tools like LangChain, LlamaIndex, MLflow, or similar.
- Experience documenting AI risks, limitations, or compliance requirements.
Click on Apply to know more.