CNTXT AI
In this remote, hourly contractor role, you will evaluate AI-generated medical content and develop cases that test clinical reasoning accuracy. Your work focuses on improving how AI models handle healthcare information by identifying errors, writing corrective feedback, and creating prompts and test cases across clinical scenarios.
Key Responsibilities
- • Evaluating AI-generated medical responses for clinical accuracy, reasoning quality, and patient safety implications
- • Identifying errors in clinical methodology, unsafe assumptions, missing contraindications, and misinterpretation of diagnostic data
- • Writing clear, precise feedback explaining corrections and reasoning gaps
- • Developing prompts and test cases that probe AI accuracy across clinical scenarios
- • Rating and comparing AI responses based on correctness, internal consistency, and contextual appropriateness
- • Fact-checking medical content against reliable sources using consistent reasoning
Required
- • Bachelor's degree or higher in Medicine (MD/DO), Nursing, Public Health, Health Sciences, or Allied Health
- • 5+ years of professional experience in a relevant healthcare discipline
- • Strong clinical reasoning skills: differential diagnosis, risk stratification, red-flag recognition
- • Solid grounding in disease processes, patient care, public health principles, and medical terminology
- • Full professional English proficiency
- • Exceptional attention to detail and ability to explain corrections clearly in writing
- • Reliable and self-directed, with consistent output quality in a remote, asynchronous workflow
Preferred
- • Experience in clinical documentation review, utilization review, or healthcare editorial QA
- • Prior experience with AI data training or annotation
Position Summary
In this remote, hourly contractor role, you will evaluate AI-generated medical content and develop cases that test clinical reasoning accuracy. Your work directly improves how leading AI models handle healthcare information, making them more accurate, reliable, and safe. Tasks may include:
Evaluating AI-generated medical responses for clinical accuracy, reasoning quality, and patient safety implications
Identifying errors in clinical methodology, unsafe assumptions, missing contraindications, and misinterpretation of diagnostic data
Writing clear, precise feedback explaining corrections and reasoning gaps
Developing prompts and test cases that probe AI accuracy across clinical scenarios
Rating and comparing AI responses based on correctness, internal consistency, and contextual appropriateness
Fact-checking medical content against reliable sources using consistent reasoning
Profile Requirements:
Bachelor's degree or higher in Medicine (MD/DO), Nursing, Public Health, Health Sciences, or Allied Health
5+ years of professional experience in a relevant healthcare discipline
Strong clinical reasoning skills: differential diagnosis, risk stratification, red-flag recognition
Solid grounding in disease processes, patient care, public health principles, and medical terminology
Full professional English proficiency
Exceptional attention to detail and ability to explain corrections clearly in writing
Reliable and self-directed, with consistent output quality in a remote, asynchronous workflow
Preferred Experience:
Experience in clinical documentation review, utilization review, or healthcare editorial QA
Prior experience with AI data training or annotation
About CNTXT AI
CNTXT AI builds artificial intelligence products and data solutions with a focus on making AI accurate, safe, and globally relevant for impact. Our work spans data services, custom AI solutions, and proprietary AI products, with deep expertise in Arabic-native and secure, sovereign solutions.
Keep track of your job search
Save personal notes for each job to track your thoughts, application status, and follow-ups.
Try for freeUpload your resume
Sign up to upload your resume and get AI-powered customization for job applications.
Sign up freePractice your interview
Get AI-powered mock interviews tailored to this Medical AI Evaluator (Remote, Hourly Contractor) role. Upload your resume and practice with real-time voice feedback.
Sign up to practice