Mirror Descent Beyond Euclidean Stability: An Exponential Separation in Initialization Sensitivity

In three linesMirror Descent with non-quadratic regularizers exhibits exponential sensitivity to initialization, unlike Gradient Descent. On smooth convex objectives with KL regularizers, initial ε perturbations amplify to min{polylog⁻¹(1/ε), εe^Ω(ηT)} after T iterations. Adding Bregman regularization toward an anchor point stabilizes dynamics.

Summary generated by Claude — human-verified