TUBE: Tangent Upper Bound on Evidence for Discrete Diffusion Language Models

In three linesTUBE is a variational upper bound on log-likelihood for discrete diffusion models. Unlike existing ELBOs, TUBE admits an unbiased Monte Carlo estimator and applies to masked diffusion models, any-order ARMs, and block variants. Experiments show discrete diffusion models lie strictly below exact ARM baselines in likelihood.

Summary generated by Claude — human-verified