Can Heterogeneous Language Models Be Fused?

In three linesHeteroFusion merges heterogeneous language models (Llama, Qwen, Mistral) by aligning functional module structures rather than raw weights, and suppressing incompatible transfer signals. Outperforms fusion, merging, and ensemble baselines on heterogeneous transfer, multi-source fusion, and cross-family generalization.

Summary generated by Claude — human-verified