Shopping Reasoning Bench: An Expert-Authored Benchmark for Multi-Turn Conversational Shopping Assistants

In three linesShopping Reasoning Bench: expert-authored benchmark of 525 missions (232 single-turn, 293 multi-turn) with 10,863 importance-weighted binary rubrics for evaluating conversational shopping assistants. Evaluation of 9 models (GPT, Claude, Gemini): pass rates 57–77%, performance degrades 4–18 points across conversation turns, 13–29 point gap between required and optional criteria.

Summary generated by Claude — human-verified