CheckMIABench: Firm Foundations For Membership Inference Attacks on Language Models

In three linesCheckMIABench introduces a benchmark for principled evaluation of membership inference attacks (MIAs) on language models. Leveraging intermediate checkpoints from open-source models (Pythia, OLMo, 70M–7B parameters), the authors construct reliable testbeds where training data before and after a fixed point share the same distribution. They evaluate six published attacks and release a modular library (pandora_llm).

Summary generated by Claude — human-verified