Side-by-side Comparison Amplifies Dialect Bias in Language Models

In three linesarXiv paper demonstrating that language models amplify dialect bias (AAVE vs Standard American English) when comparing tweet pairs side-by-side, far more than in isolated evaluation. Counterfactual fairness finetuning partially mitigates bias in isolation but fails in contrastive settings, exposing a critical gap in current evaluation frameworks.

Summary generated by Claude — human-verified