Dual-Space Knowledge Distillation with Key-Query Matching for Large Language Models with Vocabulary Mismatch

In three linesNovel DSKD-CMA-GA method for knowledge distillation between LLMs with mismatched vocabularies. Uses generative adversarial learning to align key-query distributions. Modest but consistent ROUGE-L gains (+0.37 average on out-of-distribution data).

Summary generated by Claude — human-verified