Infrastructure Engineer — Managed Inference

What we're looking for:

We need someone with 5+ years of infrastructure engineering experience who has deep hands-on expertise operating production Kubernetes at scale and working with LLM inference serving systems. You should be comfortable debugging NVIDIA GPU systems end-to-end (drivers, CUDA, NCCL, network fabric) and have a track record of building highly available, multi-cloud infrastructure for latency-sensitive AI workloads. Bonus points if you've deployed across heterogeneous accelerator types (NVIDIA GPUs, AWS Trainium, Google TPUs) or contributed to open-source inference frameworks like vLLM or SGLang.

What you'll do:

Design, operate, and scale highly available Kubernetes clusters across AWS, GCP, and specialized GPU cloud providers to serve global inference demand at 1, 000+ tokens per second

Take ownership of the production serving layer — debug NVIDIA systems end-to-end including drivers, CUDA, NCCL, node health, and network fabric

Work closely with the model optimization team to ensure kernel-level speed improvements survive contact with real traffic, real hardware, and real customers

Extend deployment tooling so models ship consistently across NVIDIA, Trainium, and TPU hosts through a unified workflow

Build observability infrastructure that keeps the platform honest: time-to-first-token, inter-token latency, throughput, and availability — measured per model, per chip, and per region

Own reliability engineering including alerting, automated failover, self-healing infrastructure, and intelligent traffic routing across models, chips, and regions