Research

Claude Fable 5 Leads New AI Medical Reasoning Leaderboard

Artificial Analysis has launched a new medical reasoning leaderboard showing that while top models can retrieve facts, most fail to synthesize complex clinical data.

AlphaSignal1 day agoResearch
Image: AlphaSignal

Artificial Analysis has introduced MLCR-AA, a new evaluation leaderboard built on Wisedocs' Medical Long Context Reasoning benchmark. The results reveal a massive gap in clinical synthesis: the median model scored below 15% on the test. Anthropic's Claude Fable 5 secured the top spot with a score of 64.4%, followed by various Claude Opus 5 configurations scoring between 53.9% and 59.4%. Among open-weights models, Moonshot's Kimi K3 (max) led its class with a score of 38.3%.

The benchmark evaluates models using 10 synthetic, real-world-inspired medical files ranging from 25,000 to 64,000 tokens, which contain 50 to 150 medical summaries. MLCR-AA tests models on 60 highly difficult questions across Expert and Compound reasoning tiers. To pass, responses must survive a conciseness check—where Nova Lite scored 100.0%—and win a majority vote from a three-model judging panel consisting of Gemini 3.1 Pro, Claude Opus 4.8, and GPT-5.5.

The evaluation highlights a critical distinction between accuracy and completeness. While GPT-5.6 Terra (max) achieved a leading 93.7% accuracy score, it finished tenth overall because it omitted crucial details. Conversely, Claude Opus 5 (Adaptive Reasoning, Max Effort) achieved the highest completeness score at 86.1%. This indicates that while modern LLMs rarely hallucinate incorrect information in long documents, they frequently fail to synthesize the entire clinical picture.

For practitioners, these capabilities come with significant financial trade-offs. Running Anthropic's top-performing models costs between $0.30 and $1.00 per task. In contrast, Kimi K3 (max) offers a budget-friendly alternative at approximately $0.15 per task, placing it on the cost-performance frontier alongside GPT-5.6 Terra and Luna. Developers building high-volume medical or insurance pipelines must weigh these costs against the necessity of comprehensive, multi-page synthesis.

This is our own summary of reporting by AlphaSignal

More in Research