EHRBench: An Automated and Reliable EHR-based Benchmark for Clinical Decision Making with LLMs

In three linesEHRBench is an automated and reliable benchmark for evaluating LLMs on clinical decision-making tasks. Built via an EHR-LLM-KB pipeline, it generates ~960k QA items covering diagnosis, treatment, and prognosis. 30+ LLMs benchmarked reveal persistent gaps toward clinical reliability.

Summary generated by Claude — human-verified