Medical AI Evaluation & Annotation

About the RoleTheora works with leading healthcare organizations and frontier AI labs to evaluate how AI models perform on real-world radiology tasks . We're looking for practicing radiologists to help build and validate high-quality radiology evaluation data using clinical context, imaging studies, radiology reports, and downstream clinical evidence.What You'll DoReview radiology cases across CT, MRI, X-ray, ultrasound, and other imaging modalitiesInterpret imaging studies using relevant clinical history and prior imagingAnnotate findings including diagnosis, localization, measurements, severity, and interval changeCreate gold-standard interpretations and reference answersEvaluate AI-generated radiology outputs for clinical accuracy, completeness, and safetyGrade model reasoning using structured clinical rubricsIdentify model failure modes such as missed findings, incorrect localization, poor comparison with priors, or inappropriate recommendationsExample TasksImage interpretation Review a CT study and determine whether the model correctly identifies and localizes clinically significant abnormalities.Longitudinal reasoning Compare current and prior imaging and determine whether the model correctly identifies progression, stability, or resolution.Clinical-context reasoning Determine whether the model appropriately incorporates the imaging indication, relevant history, and prior studies into its interpretation.Who We're Looking ForMD / equivalent medical degreeCompleted or currently completing Radiology trainingBoard-certified / board-eligible Radiologist preferredStrong experience interpreting diagnostic imagingComfortable with CT, MRI, X-ray, ultrasound, or relevant subspecialty imagingExcellent attention to detailStrong written clinical communicationAbility to explain clearly why an interpretation is correct or incorrectEspecially interested in candidates with:Subspecialty fellowship training (e.g., neuroradiology, MSK, body/oncologic imaging, thoracic, breast)Prior experience with AI/CAD tools in clinical workflowPrior research, peer review, or teaching experience (residents/fellows) — useful for producing rubric-quality gold-standard reasoning, not just answersHow to Applyemail jobs@theorahq.com with your resume!