BenchTrace: A Benchmark for Testing Reflection Ability and Controlled Evolution in LLM Agents

In three linesBenchTrace is a benchmark for evaluating self-evolution ability in LLM agents. Built on 1,821 annotated episodes across six tasks, it measures reflection quality and tests whether agents avoid past failures. Experiments on Qwen3-32B and GPT-4.1: <30% pass rate on reflection evaluation, agents forget early lessons and fail to generalize reflections.

Summary generated by Claude — human-verified