LoSoNA: A Benchmark for Local Social Norm Adaptation in Group Conversations

In three linesLoSoNA is a benchmark measuring LLM ability to recognize and adapt to local social norms in group chats. Eight frontier and open-weight models tested under four prompting conditions: Gemini 3.1 Pro reaches 84.2%, Claude Fable 5 81.6%. Explicit norm-aware prompting helps unevenly.

Summary generated by Claude — human-verified