Pocket Foundation Models: Distilling TFMs into CPU-Ready Gradient-Boosted Trees

In three linesDistillation of tabular foundation models (TabICLv2) into boosted trees (XGBoost/CatBoost) for ultra-fast CPU inference. Solves soft target collapse via stratified out-of-fold labeling. Across 153 datasets: 0.882 macro-mean AUC (96.5% of teacher) at 1.9 ms on CPU, 38–860x speedup. Open-sourced as TabTune library.

Summary generated by Claude — human-verified