AI QA & Evaluation Lead

Jodayn is seeking an AI QA & Evaluation Lead to establish and manage enterprise-grade quality assurance, validation, and evaluation processes for AI-powered systems and intelligent automation platforms.The successful candidate will be responsible for ensuring the reliability, consistency, security, explainability, and governance alignment of AI-generated outputs and AI-driven workflows.This role is critical in maintaining enterprise trust, operational quality, and compliance standards across AI initiatives delivered to government and enterprise clients.The ideal candidate must possess strong experience in enterprise quality assurance, automation testing, AI evaluation methodologies, and governance-oriented validation frameworks.RequirementsAI Evaluation & ValidationDesign and maintain AI testing and evaluation frameworksDevelop benchmarking datasets and output validation methodologiesEvaluate AI-generated responses for consistency, accuracy, and reliabilityDefine confidence scoring and quality measurement approachesAI Quality AssuranceImplement automated regression testing for AI workflowsMonitor prompt performance and behavioural consistencyDetect model drift and output degradationValidate multilingual AI outputs including Arabic and English contentGovernance & ComplianceEnsure AI outputs align with enterprise governance standardsSupport auditability and traceability of AI-generated resultsImplement review and approval workflows for AI systemsSupport compliance validation activities for enterprise deploymentsSecurity & Risk TestingConduct AI security and prompt injection testingValidate protections against unsafe outputs and misuse scenariosSupport secure AI deployment and operational practicesReporting & Continuous ImprovementProduce quality assurance reports and evaluation summariesRecommend improvements to prompts, workflows, and AI modelsSupport continuous optimisation of AI solution qualityRequired QualificationsEducationBachelor's degree in Computer Science, Information Systems, Software Engineering, or related fieldExperienceMinimum 5 years experience in enterprise quality assurance or software testingExperience with automation testing frameworksExperience evaluating AI-powered systems preferredExperience in enterprise or regulated environments preferredRequired Technical SkillsQuality AssuranceEnterprise QA methodologiesAutomation testing frameworksAPI testingRegression testing methodologiesTest case developmentAI EvaluationLarge language model evaluationPrompt engineering conceptsAI benchmarking methodologiesAI governance and validation conceptsSecurity & GovernanceRisk assessment conceptsAI security testingAuditability and traceability principlesPreferred SkillsExperience in enterprise AI initiativesExposure to regulated industriesExperience with bilingual AI validationFamiliarity with enterprise governance frameworksCore CompetenciesExceptional attention to detailStrong analytical thinkingStructured problem-solving capabilityHigh quality and governance awarenessStrong documentation and reporting capability