QA Automation Engineer

We’re Hiring: Senior QA Automation Engineer – AI/LLM (Fully Remote)📍 Abu Dhabi, UAE📄 12 months extendable contractWe are looking for a Senior QA Automation Engineer to help define how advanced AI systems are tested, validated and approved for deployment within a highly regulated environment.This is not a traditional QA role. You will build automated frameworks to assess LLM and RAG outputs for hallucination, factual accuracy, consistency, faithfulness and answer relevance.Key responsibilities:• Build Python-based test automation frameworks using Pytest• Evaluate LLM outputs against ground-truth datasets• Implement RAG evaluation metrics using RAGAS, DeepEval, TruLens or custom evaluators• Develop prompt-regression testing to identify model and prompt drift• Automate UI and API testing using Playwright/Selenium and Requests• Validate integrations between AI agents and enterprise systems• Test the latency and throughput of AI and predictive-model services• Verify API security, RBAC and PII-redaction controls• Validate data quality across SQL databases, data lakehouses and vector databases• Integrate automated tests into GitLab CI/CD quality gates• Produce traceable evidence for formal verification and release reviews• Work closely with data scientists, developers and systems engineersWhat we’re looking for:• 5+ years of QA automation experience• At least 2 years testing AI/ML systems, data-driven applications or AI agents• Strong Python, Pytest, SQL and API automation experience• Experience testing LLMs, RAG pipelines or probabilistic systems• Knowledge of ground-truth datasets and AI evaluation metrics• Experience with GitLab CI/CD and automated quality gates• Familiarity with Jira, Jama, Polarion or similar traceability toolsInterested or know someone suitable? Please message me directly or send your CV.