AI Test Engineer - GO/JC/4683/2026

Job title: AI Test Engineer

Location: Pan IndiaWork Mode: RemoteCandidate Specification: 3+ years overall, Relevant – 3+ yearsCandidate Location: Pan IndiaSkills: MCP, RAG, agentic AI, LangSmithCandidate Industry: ITEducation and Certification: Any GraduationJob DescriptionMust have 3+ years in software QA/testing, with hands-on senior-level experience evaluating LLM-based or agentic AI systems (not just traditional QA)Direct experience with eval frameworks/tools — LangSmith strongly preferred; also acceptable: Ragas, DeepEval, PromptFoo, Braintrust, LangfuseStrong understanding of RAG architecture and how to independently evaluate retrieval quality vs. generation qualityPractical familiarity with MCP (Model Context Protocol) or comparable agent tool calling frameworksStrong Python skills for building test harnesses, scoring pipelines, and automationExperience designing evaluation approaches for non-deterministic systems (probabilistic scoring, not binary pass/fail)Comfortable working directly with client engineering teams — strong communication, able to explain quality risk to both technical and business stakeholders