Website:
appzime.com
Job details:
Principal Accountabilities
1. Production‑grade prompting & orchestration
● Ship robust prompt chains using few‑shot, chain‑of‑thought (CoT), self‑consistency
CoT, least‑to‑most, and variants (e.g., debate/Tree‑of‑Thought).
● Implement guardrails and structured output (JSON) contracts; integrate retries,
validators, and fallbacks.
2. LLM sampling strategy design & tuning● Tune temperature, top‑k/top‑p (nucleus), beam search, repetition penalty, max
tokens, and multi‑pass/self‑consistency sampling.
3. Error analysis & mitigation
● Identify and mitigate hallucinations, instruction drift, context leakage, prompt
fragility, and format non‑compliance.
4. Evaluation & experimentation
● Set up prompt evaluation suites with metrics: accuracy, pass@k, win‑rate vs.
baselines, faithfulness, format compliance, toxicity/safety, latency, and cost.
5. Tools & platform integration
● Build prompt pipelines in Python; integrate with LangChain, Pinecone, and prompt
evaluation tools.
6. Domain collaboration & requirements mapping
● Work closely with Product and Business teams to translate tower operations rules
into efficient prompt policies.
7. Research tracking & continuous improvement
● Stay current with state‑of‑the‑art prompting and LLM/VLM research; introduce
relevant techniques to improve robustness and reduce cost.
Required Qualifications & Skills
● 3-4 years of production experience designing advanced prompts: few‑shot, CoT,
self‑consistency CoT, least‑to‑most.
● In‑depth knowledge of LLM sampling strategies and trade‑offs.
● LLM error taxonomy & mitigation strategies.
● Up‑to‑date with latest research and state‑of‑the‑art prompting techniques.
● Proficient in Python for rapid prompt iteration.
● Hands‑on with LangChain, Pinecone, and prompt evaluation tools.
● Strong debugging skills.
● Collaboration and communication skills.
Preferred / Good‑to‑Have
● Experience with Vision‑Language Models and OCR+LLM pipelines.
● Familiarity with guardrails & safety tooling.
● Exposure to Azure/OpenAI/Hugging Face ecosystems.
● Background in Computer Vision (OpenCV, YOLOv8, segmentation, OCR).
● Experience with A/B testing and statistical analysis.KPIs (Indicative)
Accuracy vs. baseline, latency reduction, cost per task, format compliance rate, incident rate,
throughput of evaluated experiments
Click on Apply to know more.