Senior AI Evaluation Engineer

Senior AI Evaluation Engineer (Data Scientist) - London/Manchester/Bristol The Incubator for AI (i.AI) is a team of around 80 builders and operators using frontier AI to improve public services across healthcare, education, planning and parliament. We work at startup pace, with the backing of the Prime Minister and No10, on problems that affect millions of people.Our Impact and Evaluation team embeds within product teams to build rigorous evaluation into every stage of development, shaping product decisions and showing whether our products improve public services. We’re now hiring two Senior AI Evaluation Engineers to focus on model-level evaluation of accuracy, bias and safety.What will I be doing?Leading the design of evaluation pipelines and architecting product-wide evaluation suites that integrate with engineering and CI/CD workflowsDeveloping complex and exploratory methods to assess accuracy, bias, safety and model performanceWorking with product, engineering and domain experts to shape evaluation across the product lifecycle and inform technical decisionsTurning complex results into clear recommendations and leading communication of findings through reports, presentations and external publicationsReviewing and supporting other engineers’ work to raise standards and build evaluation capability across the teamWhat will I need?You don’t need to tick every box. You’ll likely bring some of the following:Broad experience evaluating generative and predictive AI systems, including LLM-as-a-judge and tools such as pydantic evals, deepeval or inspectExtensive knowledge of experimental design, statistical hypothesis testing and quantitative research methods (particularly those in the Magenta Book)High proficiency in Python and strong software practices, including git, CI/CD, unit testing and pipeline orchestration tools like DVC or snakemakeThe ability to communicate technical work to broad audiences through reports, presentations and web-based toolsExperience mentoring junior colleaguesWhat’s in it for me?Career-defining work. Projects backed by No10 that apply AI to public services affecting millions, alongside talented, mission-driven colleaguesThe tools to ship. Frontier models, ample compute, access to experts in national security, policy and AI research, and full operational and delivery supportEarly ownership of important products in small teams, 5 L&D days a year, and funding for learning, conferences and collaborations£63,165–£86,640 in London (£58,275–£81,885 nationally), including a technical allowance of up to £15,600, plus a 28.97% employer pension contributionFlexibility and benefits. Hybrid working from London, Manchester or Bristol, 25+ days’ leave, up to 39 weeks’ full-pay parental leave, and perks including cycle to work and dental insuranceApply now for immediate consideration.