Neural Estimation of Pairwise Mutual Information in Masked Discrete Sequence Models

In three linesNeural framework to estimate pairwise conditional mutual information directly from hidden states of masked diffusion models (MDMs). The estimator captures the model's internal dependency structure and enables MI-guided parallel decoding, reducing inference forward passes by 3-5x on Sudoku and protein sequence generation (ESM-C) while preserving quality.

Summary generated by Claude — human-verified