Randomized Advantage Transformation (RAT): Computing Natural Policy Gradients via Direct Backpropagation

In three linesRAT (Randomized Advantage Transformation) estimates Tikhonov-regularized natural policy gradients via direct backpropagation without explicit Fisher matrix construction. The method applies the Woodbury formula and randomized block Kaczmarz iterations on on-policy mini-batches. Results match or exceed established natural-gradient methods on continuous and visual control benchmarks.

Summary generated by Claude — human-verified