SciCustom: A Framework for Custom Evaluation of Scientific Capabilities in Large Language Models

In three linesSciCustom is a framework for building custom benchmarks to evaluate application-specific scientific capabilities in LLMs. It organizes scientific knowledge into ontology-grounded units, uses multi-model consensus voting to identify relevant units, and generates benchmarks from real data in chemistry and healthcare without expert annotation.

Summary generated by Claude — human-verified