Where you can live
Open to people living in United States.
Listed location: San Francisco, CA, New York, NY (Remote)
Taken from the location the company listed. Confirm in the posting before you apply.
What the job is
Owns and scales the inference gateway that routes every model call, building the control plane for model selection, fallback, cost metering and enforcement.
What they ask for
- Hands-on production experience with model serving stacks such as vLLM or SGLang
- Understanding of continuous batching, KV and prefix caching, and time to first token tradeoffs
- Fluency across provider APIs, including prompt-caching and cached-token usage reporting
- Experience building or operating a proxy, gateway, or data plane on production traffic
- Strong backend generalist skills in Python and Kubernetes
Nice to have
- Experience at an inference provider or GPU cloud, or open-source serving stack contributions
- Fine-tuning and evaluating models in production
- Usage metering, billing, or cost optimization at scale
This is a summary written by software from the company's posting. The posting is the source.
More remote jobs at You.com