Innova ESI
Website:
innovaesi.com
Job details:
PAN India Location
Hybrid/Remote
About the Role
We are looking for a Senior Apigee Engineer with deep hands-on experience building and operating Apigee as an AI Gateway to join our enterprise platform team. This is not a general API management role — you will own the design, implementation, and governance of the layer that sits between our AI applications and every LLM they call, whether those models live on Vertex AI, Azure OpenAI, or third-party providers.
You will be the go-to engineer for the hardest problems at the intersection of API management and generative AI: semantic caching to reduce token spend, LLM circuit breaking to handle provider outages, multi-cloud model routing to optimize cost and latency, and prompt/response sanitization to keep the enterprise safe. The work you build will directly govern how every AI agent in the organization reaches its tools and models.
What You Will Do
AI Gateway Design and Implementation
- Design and implement Apigee API proxies purpose-built for LLM traffic, including token-aware rate limiting, quota enforcement, and model abstraction layers that decouple applications from specific model providers
- Build multicloud LLM routing logic that dynamically routes requests between Vertex AI, Azure OpenAI, AWS Bedrock, and self-hosted models based on latency, cost, quota availability, and failover state
- Implement the LLM circuit breaker pattern to prevent cascade failures when upstream model providers return 429s or experience degraded throughput — ensuring high availability for production AI applications
- Configure semantic caching using vector search and prompt proximity scoring to reduce redundant LLM calls and cut token spend across the platform
- Build request and response enrichment flows that inject dynamic context (RAG retrieval results, user profile data, session state) into LLM requests before they reach the model
Agentic Workflow Integration
- Design Apigee as the intermediary layer for multi-agent systems built on LangChain, LangGraph, LlamaIndex, and Google Cloud Agentspace — managing authentication, authorization, and tool connectivity for every agent-to-system call
- Integrate Apigee with Google Cloud Application Integration to enable agent connectivity to third-party enterprise systems (Salesforce, ServiceNow, SAP, and internal APIs)
- Enforce token budget limits per agent, per session, and per model tier to control generative AI costs at the infrastructure layer rather than relying on application-level controls
- Catalog first-party and third-party AI APIs and agent tools in Apigee API Hub, maintaining enterprise context for consistent API specification generation and duplicate detection
AI Governance and Observability
- Build and maintain policy-based access controls for model access — governing which applications, teams, and personas can call which models at what rate, enforced at the gateway rather than in application code
- Design token usage dashboards in Looker Studio integrated with Apigee observability data, providing platform teams and cost owners with per-application, per-model, and per-team token consumption reporting
- Implement LLM audit logging pipelines that capture prompt and response data (redacted per data classification policy) for compliance, anomaly detection, and model performance analysis
- Configure and tune Apigee Advanced API Security for AI workloads — including abuse detection tuned to LLM traffic patterns, misconfiguration scanning, and integration with Google SecOps and enterprise SIEM
AI Safety and Security
- Integrate Google Cloud Model Armor into Apigee proxy flows for prompt injection detection, toxic content filtering, and PII sanitization before requests reach LLM endpoints and after responses are returned
- Implement OWASP LLM Top 10 mitigations at the gateway layer — covering prompt injection (LLM01), insecure output handling (LLM02), sensitive information disclosure (LLM06), and excessive agency (LLM08)
- Design and enforce OAuth 2.0, API key, and JWT-based authentication flows for all LLM API consumers, with scoped authorization that restricts access to specific models and capabilities by application type
- Build anomaly and abuse detection policies tailored to LLM traffic — detecting prompt flooding, token exhaustion attacks, and systematic prompt injection attempts at gateway throughput rather than at the model layer
Platform Engineering
- Manage Apigee X or Apigee hybrid deployments on Google Cloud, including OpenShift-based deployments where required, and maintain CI/CD pipelines for proxy deployment using Apigee Maven Plugin, apigeecli, or Terraform
- Define and enforce API design standards across the organization using Apigee API Hub with Gemini Code Assist for spec generation, ensuring new AI APIs conform to enterprise standards before they reach production
- Collaborate with AI/ML engineers, security engineers, and enterprise architects to evolve the AI gateway architecture as new model providers, agent frameworks, and governance requirements emerge
Required Experience
- 5+ years of hands-on Apigee experience in production environments, with at least 2 years on Apigee X or Apigee hybrid (not just Apigee Edge)
- Direct experience implementing Apigee as an AI Gateway or LLM proxy — you have built at minimum one of: semantic caching, LLM circuit breaking, multi-model routing, or token-aware rate limiting in a real production environment
- Strong command of Apigee policy configuration: AssignMessage, ExtractVariables, JavaScript callouts, ServiceCallout, RaiseFault, SpikeArrest, Quota, OAuthV2, and VerifyAPIKey — and knowing when to use each
- Experience integrating Apigee with Vertex AI, OpenAI-compatible APIs, or other LLM endpoints, including handling streaming responses (SSE/chunked transfer) correctly through the gateway
- Proficiency in at least one infrastructure-as-code tool for Apigee deployments: Terraform (google_apigee_* resources), apigeecli, or Apigee Maven Plugin within a CI/CD pipeline (GitHub Actions, Cloud Build, Jenkins)
- Working knowledge of OAuth 2.0, JWT, and API key flows — both the configuration side in Apigee and the integration side with identity providers (Azure Entra ID, Okta, Google Cloud Identity)
- Solid understanding of LLM fundamentals relevant to gateway operations: token counting, prompt/completion structure, streaming vs. non-streaming, context window management, and how these affect rate limiting and caching strategy
Strongly Preferred
- Experience with Google Cloud Model Armor for prompt injection and content safety policy enforcement within Apigee flows
- Familiarity with OWASP LLM Top 10 and the ability to map gateway-level controls to each risk category
- Hands-on experience with Apigee API Hub for API cataloging, lifecycle management, and spec generation with Gemini Code Assist
- Experience integrating Apigee with Google Cloud Application Integration for agent-to-enterprise-system connectivity
- Knowledge of semantic caching patterns using vector similarity search (Vertex AI Vector Search, Milvus, or similar) applied to LLM prompt deduplication
- Experience building observability pipelines for LLM traffic: token usage reporting, cost attribution by team or application, and anomaly detection on prompt/response patterns
- Prior work on multi-agent architectures using LangChain, LangGraph, or LlamaIndex where Apigee served as the tool gateway
- Google Cloud Professional Cloud Developer or API Engineer certification, or equivalent demonstrated depth
Nice to Have
- Experience with Apigee on RedHat OpenShift (relevant to hybrid enterprise deployments)
- Familiarity with Google SecOps or enterprise SIEM integration for LLM audit log forwarding
- Exposure to IBM WatsonX Orchestrate or other enterprise agent runtimes that use Apigee as an MCP-compatible tool gateway
- Understanding of RAG architecture and how to enrich LLM requests with retrieval context inside an Apigee flow before the request reaches the model
What You Will Not Be Doing
This role is not general API management maintenance, portal administration, or legacy Apigee Edge migration work. If you are primarily a REST API designer or an API product manager who uses Apigee as a traffic router, this is not the right fit. The role requires genuine depth in LLM-specific gateway patterns — the people you will be working with are building production AI agents and they need the gateway layer to be as thoughtfully engineered as the agents themselves.
Our Stack Context
Our AI agents run on IBM WatsonX Orchestrate on RedHat OpenShift (GCP). Apigee X is the centralized API gateway and MCP policy enforcement layer for all 19 onboarding agents, handling authentication, rate limiting, model routing, and observability. You will be working directly with the agent engineering team to evolve gateway policy as new agents, tools, and model providers are added to the program.
Click on Apply to know more.