Staff AI Product Engineer

Staff AI Product EngineerThe OpportunityOur Backend Engineering team builds the highly scalable distributed services that power a fast-growing ML observability platform. While Go is the primary language for these distributed systems, the team also maintains services and tools written in Python, Java, and TypeScript. As a Staff engineer, you'll set the technical direction for how the team tackles its hardest problems — whether that's re-architecting the OLAP database layer for another order of magnitude of scale, designing the evaluation infrastructure that underpins how customers monitor LLM and agentic systems, or pushing the boundaries of dimensionality reduction and visualization techniques in a distributed environment.You'll operate at the intersection of deep technical craft and product judgment. You will shape how some of the most impactful engineering teams in the industry build and monitor AI and LLM-powered applications, and you'll be trusted to make the architectural calls that determine whether the platform can keep pace with them — from clean APIs that instrument applications in seconds, to real-time evaluation infrastructure processing millions of annotations per second.What You'll DoSet technical direction and architecture for core backend systems, primarily in Go, Java, and Python, with scope extending across TypeScript services where needed.Lead the design of high-volume, highly available analytics systems, owning tradeoffs around scale, cost, and reliability at the platform level.Architect APIs and data models that serve customers' most demanding ML and LLM workflows, anticipating needs before they become bottlenecks.Drive major technical initiatives end-to-end: prototyping, de-risking, and scaling backend services that power the core platform.Extend and contribute back to open source OLAP databases and distributed message queue frameworks, representing the company in the broader systems community where relevant.Define the technical strategy for collection tooling that monitors ML and LLM pipelines at scale.Research and productionize cutting-edge visualization and dimensionality reduction algorithms in distributed environments.Partner directly with product, design, and customer engineering teams as a technical thought partner, shaping product direction as much as implementation.Mentor senior and mid-level engineers, raise the technical bar across the team, and help instill strong engineering practices.Contribute to and help guide the build-out of the company's in-house AI Agents.What We're Looking For8+ years of experience building high-performance backend systems, with a track record of owning architecture for complex, large-scale platforms.Deep expertise in Go, Python, TypeScript/Node, Java, or similar server-side languages, with the judgment to choose the right tool for the problem.A track record of technical leadership — driving cross-team initiatives, setting standards, and influencing engineering direction beyond your immediate team.Genuine enthusiasm for the AI and LLM ecosystem, with a habit of staying ahead of emerging techniques and translating them into product.Extensive experience building and operating highly complex SaaS platforms/systems at scale.Strong command of public cloud infrastructure and container orchestration — AWS, GCP, Azure, Kubernetes, etc.Bonus Points, But Not RequiredDeep experience with distributed stream processing — Kafka, Gazette, or similar.Hands-on experience architecting or operating OLAP systems.Familiarity with system observability tooling like Prometheus.Strong working knowledge of Machine Learning and/or Data Science.First-hand experience building AI products or working directly with large language models.The company is remote-first, with offices in New York City and the San Francisco Bay Area for those who wish to work in-person. For all other employees, there is a WFH monthly stipend to pay for co-working spaces.