Machine Learning Specialist - Inference

Staff / Principal Machine Learning Engineer, ServingRemote (US/UK/Canada/Germany/Switzerland/Serbia)$270K - $500K + competitive equityClient provides the world's best realtime AI models for consumer-scale applications. Its core product suite includes:Realtime Voice Models — the world's #1 ranked text-to-speech (TTS) and speech-to-text (STT) models, engineered for low-latency, production-grade voice experiencesSpeech-to-Speech API — a Realtime API enabling seamless, natural conversational AI interactions at scaleWho We're Looking Forwe're looking for strong people from varied backgrounds who learn fast, thrive in ambiguity, and can show us what they've built, broken, and understood.Experience We Find UsefulYou don't need all of this. But you need enough to make a case.Inference Optimization. Deep understanding of modern serving frameworks and techniques like vLLM or TRT-LLM.Model Acceleration. Hands-on experience with quantization, distillation, caching strategies , continuous batching, paged attention, and speculative decoding.High-Performance Systems. Proficiency in C++, CUDA, Rust, or highly optimized Python. YDistributed Systems & Scaling.Experience with Kubernetes, Ray, custom load balancing, multi-GPU/multi-node inference, and reliably handling thousands of concurrent connections.You can take a model from the research team, containerize it, optimize its serving, and ensure it runs reliably in production.Background. PhD in CS, Physics, Math, or equivalent practical experience building backend or ML systems.H-1B transfers supported. Relocation assistance available for candidates willing to move to the SF Bay Area.Remote work policyTech stackvLLM, TRT-LLM, CUDA, C++, Rust, Python, Kubernetes, Ray, NVIDIA GPUs, Docker