Concentric AI
Website:
concentric.ai
Company:
https://www.linkedin.com/company/concentricinc
Seniority: Mid-Senior level
Industries: Software Development
Job details:
About the Role
We are building the inline cloud data plane that powers a next-generation AI security platform. This service terminates TLS, inspects and processes enterprise AI traffic in real time, and applies policy, classification, orchestration, auditing, and analytics — at millisecond latencies under production SaaS load across multiple global regions.
We are looking for a hands-on Principal Engineer to own the architecture, design, and implementation of this data plane end to end.
What You’ll Do
- Architect and build the inline cloud data plane that processes enterprise AI traffic in real time.
- Own the request path end to end: TLS termination, HTTP/2 and HTTP/3 handling, policy evaluation, classification hooks, upstream routing, and response streaming.
- Design multi-tenant, multi-region PoP architecture with strict p50 / p95 / p99 latency budgets and availability SLOs.
- Design and implement request routing, service discovery, resiliency, timeout, retry, circuit-breaking, and backpressure strategies on the hot path.
- Optimize the platform for added latency, throughput, connection density, TLS handshakes/sec, and cost per million requests.
- Drive cost and operational efficiency across the data plane, including capacity planning, autoscaling behavior, infrastructure utilization, and cost per unit of traffic.
- Integrate the data plane with policy enforcement, classification, orchestration, auditing, analytics, and other core platform services.
- Define the control-plane integration model for configuration distribution, policy propagation, service discovery, tenant isolation, and safe rollout of data-plane changes.
- Build distributed tracing, metrics, and end-to-end observability tuned for inline request-path debugging.
- Architect Kubernetes-based deployments, horizontal autoscaling, rolling upgrades, and multi-region disaster recovery for the data plane.
- Drive technical decisions on proxy technology selection (Envoy vs. custom vs. hybrid), extension model (filters, WASM, native), and control-plane integration.
- Mentor engineers and set the engineering bar for inline-service development, incident response, and operational excellence.
Required Qualifications
- 10+ years of software engineering experience, with at least 5 years building or operating an inline, customer-traffic, low-latency SaaS data plane (proxy, gateway, edge, CDN, SASE/SWG, or equivalent).
- Production experience with a programmable L7 proxy — Envoy, Nginx/OpenResty, HAProxy, Squid, or a comparable in-house proxy — including writing filters, modules, or extensions.
- Deep hands-on expertise with HTTP/1.1, HTTP/2, HTTP/3 (QUIC), gRPC, WebSockets, TLS (termination, MITM/SSL-inspection, SNI, session resumption, cert management at scale), and streaming architectures.
- Proven track record of hitting hard SLOs on a live data plane: peak RPS, p99 added latency, TLS handshakes/sec, concurrent connections — you can quote your numbers.
- Strong expertise in multi-tenant, multi-region service design, including PoP architecture, global load balancing, and failover.
- Deep understanding of Kubernetes and cloud-native application design as it applies to inline, stateful / long-lived-connection workloads.
- Strong systems-programming skills in Go (preferred), Rust, C++, or Java.
- Experience with observability platforms — OpenTelemetry, Prometheus, Grafana, or Datadog — for high-cardinality, request-path observability.
- Excellent architectural, debugging, and performance-optimization skills under production load.
Preferred Qualifications
- Experience building or extending Envoy (filters, xDS control plane), API gateways (Kong, Apigee, Tyk), service mesh (Istio, Linkerd, Consul Connect), or edge/CDN platforms.
- Production experience with SWG, SASE, CASB, ZTNA, or forward/reverse proxy products.
- Experience with AI/LLM traffic patterns (streaming responses, long-lived SSE/WebSocket connections, token-level inspection).
- Experience designing systems supporting high-throughput traffic processing, horizontal autoscaling under sudden load, graceful degradation, and per-tenant fault isolation.
Click on Apply to know more.
