GateGPT: 56k tokens per second Transformer (KV cache) on FPGA at 80 MHz

In three linesGateGPT achieves 56k tokens/sec on FPGA at 80 MHz by optimizing Transformer KV cache. Hardware acceleration demonstration for inference.

Summary generated by Claude — human-verified