SentinelBench: A Benchmark for Long-Running Monitoring Agents

In three linesSentinelBench is an open-source benchmark for evaluating AI agents on long-running monitoring tasks (minutes to hours). It contains 100 tasks across 10 synthetic web environments (email, calendars, finance, professional networking). The benchmark measures reaction time, resource usage, and task completion, exposing the tradeoff between responsiveness and cost.

Summary generated by Claude — human-verified