Gradient Transformer: Learning to Generate Updates for LLMs

In three linesGradient Transformer, a data-free knowledge distillation framework, generates LLM update vectors from TinyLMs fine-tuned on private data. The model captures correlation between gradient vectors of both models, enabling collaborative adaptation without accessing sensitive data.

Summary generated by Claude — human-verified