TPU Performance Engineer

About the RoleWe are looking for a TPU Performance Engineer to optimize large-scale LLM inference performance on Google TPU. You will work across TPU, kernels, compilers, and runtime systems, improving inference latency, throughput, and overall efficiency.Key ResponsibilitiesOptimize LLM inference workloads on Google TPU.Develop and optimize TPU backends, kernels, compiler integrations, and runtime components.Optimize performance-critical workloads such as Attention, GEMM, KV Cache, Sampling, and fused kernels.Work with JAX, XLA, Pallas and related compiler/runtime technologies.Build benchmarking and profiling infrastructure and identify bottlenecks across compute, memory, compilation, and runtime.Collaborate with model, inference, compiler, and hardware teams to improve production performance.RequirementsBachelor’s degree or equivalent experience in Computer Science, Engineering, Machine Learning, Systems, or a related field.Hands-on experience with TPU performance optimization, preferably with JAX, XLA, Pallas, or related technologies.Strong understanding of TPU architecture, memory behavior, compilation, and ML workload performance.Experience with ML kernel optimization, LLM inference, backend/runtime development, or performance engineering.Strong C++ and/or Python programming skills.Solid experience with performance profiling and benchmarking.Preferred QualificationsExperience with vLLM, SGLang, TensorRT-LLM or other LLM inference frameworks.Familiarity with LLM serving, batching, KV Cache, decoding, and inference optimization.Experience with MLIR, LLVM, Pallas, XLA or other compiler technologies.Knowledge of FP8, INT8, mixed precision, or quantization.Contributions to vLLM, JAX/XLA, Pallas, PyTorch/XLA or other open-source AI infrastructure projects.