Neural Estimation of Pairwise Mutual Information in Masked Discrete Sequence Models

In three linesNeural method to estimate pairwise conditional mutual information in masked diffusion models (MDMs). Framework uses hidden states from pretrained MDMs with supervision from ground-truth MI computed from model's conditional distributions. Applied to Sudoku and protein sequence generation (ESM-C), reduces inference forward passes by 3-5x via MI-guided parallel decoding while outperforming entropy-based methods.

Summary generated by Claude — human-verified