Can AI Agents Synthesize Scientific Conclusions?

In three linesSciConBench, a benchmark of 9.11K questions from systematic reviews, evaluates AI agents' ability to synthesize scientific conclusions. Among 8 frontier models tested in controlled settings, the best agent achieves only 0.337 factual F1. Consumer-facing agents (Google AI Overview, OpenEvidence) frequently generate incomplete or contradictory conclusions.

Summary generated by Claude — human-verified