MedicalBench: Evaluating Large Language Models Toward Improved Medical Concept Extraction

In three linesMedicalBench is a benchmark for extracting implicit medical concepts from electronic health records (MIMIC-IV). It formulates the task as verification of note-concept pairs with sentence-level evidence identification. State-of-the-art LLMs show modest performance, highlighting the difficulty of implicit medical reasoning.

Summary generated by Claude — human-verified