Online Agent-as-a-Judge: Situation-Generating Evaluation for Interactive Agents

In three linesNew evaluation method for social AI agents: an evaluator agent actively interacts with the target agent to generate situations testing specific social criteria (conflict handling, etc.). Tested on 32 criteria in a life-simulation environment, improves coverage and agreement with human labels compared to passive methods.

Summary generated by Claude — human-verified