Revisiting Padded Transformer Expressivity: Which Architectural Choices Matter and Which Don't

In three linesTheoretical study of padded transformer expressivity. Authors prove polynomially padded constant-precision transformers are equivalent to L-uniform AC⁰, while growing-precision ones achieve L-uniform TC⁰. Model depth and numeric precision are key factors; width beyond logarithmic does not increase expressivity.

Summary generated by Claude — human-verified