The Hartford
Website:
thehartford.com
Job details:
IND Staff Engineer, Reliability - GCC070
We’re determined to make a difference and are proud to be an insurance company that goes well beyond coverages and policies. Working here means having every opportunity to achieve your goals – and to help others accomplish theirs, too. Join our team as we help shape the future.
Position Summary
We are seeking a highly skilled T7 AI Operations & Site Reliability Engineer to join our engineering team in Hyderabad, India. This role is laser-focused on the availability, reliability, and performance of our production AI systems. You will own the operational health of AI-powered products — ensuring LLM-based services, agentic workflows, RAG pipelines, and ML inference platforms maintain enterprise-grade uptime while scaling to meet demand. You will build the observability, automation, and incident response capabilities that keep our AI products running 24/7.
Level: T7 (Senior Engineer)
Location: Hyderabad, India
Employment Type: Full-Time
Key Responsibilities
AI Platform Reliability & Uptime
- Own the end-to-end reliability of production AI systems including LLM services, RAG pipelines, agentic workflows, and inference endpoints
- Define and maintain SLOs/SLIs/SLAs for AI products — latency, availability, error rates, token throughput, and response quality
- Design and implement high-availability architectures for AI workloads: multi-region failover, load balancing, auto-scaling, and graceful degradation
- Build circuit breakers, retry logic, fallback models, and rate-limiting strategies to ensure AI services remain available under stress
- Drive availability targets of 99.9%+ for critical AI-powered products
- Establish disaster recovery procedures and regularly test backup/restore for AI data stores and model artifacts
Observability & Monitoring
- Build and maintain comprehensive observability stacks for AI systems — metrics, logs, traces, and AI-specific signals (hallucination rates, model drift, token costs)
- Implement real-time dashboards and alerting for AI service health, model performance, and infrastructure utilization
- Design anomaly detection and proactive alerting to identify degradation before users are impacted
- Monitor LLM provider dependencies (GCP Vertex AI, OpenAI, etc.) and implement automated failover when external services degrade
- Track and optimize cost-per-inference, token utilization, and resource efficiency across AI workloads
Incident Management & Response
- Lead incident response for AI system outages and degradations — triage, mitigate, resolve, and communicate
- Build and maintain runbooks for common AI failure modes: model timeouts, context window overflows, embedding pipeline failures, vector DB issues
- Establish on-call rotations and escalation procedures tailored to AI system failure patterns
- Automate incident detection and remediation where possible — self-healing pipelines and auto-rollback
AI Infrastructure & Platform Operations
- Operate and scale cloud-native AI infrastructure (GCP, AWS) including model serving platforms, Kubernetes containers, and vector databases
- Implement and maintain Infrastructure-as-Code (Terraform) for AI platform environments
- Automate deployment pipelines for model updates, configuration changes, and infrastructure scaling
Collaboration & Documentation
- Partner closely with AI Engineers to ensure new features are built with operability, observability, and reliability in mind
- Define production-readiness criteria for AI services — ensuring all systems meet reliability standards before launch
- Maintain comprehensive operational documentation: architecture diagrams, runbooks, playbooks, and SOPs
- Contribute to architecture reviews with a reliability lens — identifying single points of failure, blast radius, and operational risk
- Participate in on-call rotations and drive continuous improvement of operational practices
Required Qualifications
- Experience: 8+ years of professional experience in software engineering, DevOps, or site reliability engineering, with 1+ year operating AI/ML systems in production
- Education: Bachelor's degree in Computer Science, Software Engineering, or related field (or equivalent experience)
- SRE Fundamentals:
- Deep understanding of SRE principles: SLOs, error budgets, toil reduction, incident management, and capacity planning
- Proven track record maintaining high availability (99.9%+) for production systems at scale
- Experience integrating into observability platforms (Prometheus, Grafana, Datadog, Splunk, or equivalent)
- Strong incident response skills with experience leading war rooms and post-incident reviews
- AI/ML Operations:
- Understanding of AI-specific failure modes: model drift, hallucination spikes, token limit errors, embedding pipeline failures, and provider outages
- Familiarity with LLM providers and platforms (GCP Vertex AI, OpenAI, AWS Bedrock) from an operational perspective
- Cloud & Infrastructure: Advanced-level experience with cloud platforms (GCP, AWS), Kubernetes, containerization, and Infrastructure-as-Code (Terraform)
- Programming: Strong proficiency in Python and at least one systems language; comfortable writing automation scripts, custom exporters, and operational tooling
- Networking & Security: Solid understanding of networking, load balancing, DNS, TLS, and security best practices for cloud-native systems
- CI/CD: Experience building and maintaining deployment pipelines (Jenkins, GitHub Actions, ArgoCD) with automated rollback capabilities
- Communication: Excellent communication skills for incident coordination, stakeholder updates, and cross-team collaboration
Preferred Qualifications
- Knowledge of AI cost optimization strategies — model routing, caching, batching, and tiered inference
- Experience in regulated industries (insurance, finance, healthcare) with compliance and audit requirements
- Cloud certifications (GCP Professional Cloud Architect, AWS Solutions Architect, CKA/CKAD)
- Experience with AIOps — using AI/ML to improve operational intelligence and automated remediation
Click on Apply to know more.
