POLAR-Bench: A Diagnostic Benchmark for Privacy-Utility Trade-offs in LLM Agents

In three linesPOLAR-Bench is a diagnostic benchmark assessing privacy-utility trade-offs in LLM agents. A trusted model with privacy policy interacts with an adversarial third-party model across 10 domains and 7,852 samples. Frontier models withhold 99% of protected attributes, but open-weight models in the 1–30B range commonly used for on-device private inference leak up to 50% of sensitive data.

Summary generated by Claude — human-verified