Taxonomy and Consistency Analysis of Safety Benchmarks for AI Agents

In three linesSystematic analysis of 40 agent safety benchmarks (2023-2026). Benchmarks exhibit incompatible threat models, fragmented metrics, and inconsistent risk coverage. Concordance test (Kendall's W = 0.10, p = 0.94) reveals no ranking alignment across evaluation dimensions. Releases structured metadata and proposes minimum reporting standards.

Summary generated by Claude — human-verified