BitCPM-CANN: Native 1.58-Bit Large Language Model Training on Ascend NPU

## BitCPM-CANN: Native Ternary Training on Ascend NPU — What Actually Matters

### 1. What's happening

OpenBMB (Tsinghua) releases BitCPM-CANN, a complete 1.58-bit (ternary) quantization-aware training pipeline ported natively to Huawei Ascend NPUs via CANN, MindSpeed, and Megatron-LM. Four models — 0.5B, 1B, 3B, 8B — are trained from scratch with strict architectural parity to MiniCPM4, on identical pre-training data. This is not post-training quantization or fine-tuning: ternary weights {-1, 0, +1} are learned from the start through QAT.

Core result: the 1B, 3B, and 8B variants retain **95.7–97.2%** of full-precision performance across 11 benchmarks covering commonsense reasoning, domain knowledge, mathematics (GSM8K), and complex reasoning (BBH). The 3B variant reaches parity on BBH; the 3B and 8B variants recover nearly all of GSM8K. Only the 0.5B drops to 90.1%, and the authors explicitly identify model capacity — not the quantizer — as the bottleneck below one billion parameters.

### 2. The numbers that matter

**Training overhead: 4.5%** — 148 vs. 155 TFLOP/s per NPU. This is the most operationally significant figure in the paper. A 4.5% overhead means ternary training becomes viable as a **default configuration**, not an exceptional trade-off. Prior QAT pipelines on GPU showed substantially higher overheads once you left optimized CUDA kernels behind.

**Memory reduction: 8× on weights, ~6× end-to-end** (including scaling factors). For an 8B model, this brings weight memory from ~16 GB (BF16) to ~2 GB, with under 5% performance degradation. No post-training quantization technique — GPTQ, AWQ, GGUF Q2 — achieves this ratio at this retention level on reasoning tasks.

**MiniCPM4 context**: the full-precision 8B base model matches Qwen3-8B trained on 36 trillion tokens using only 8 trillion tokens. BitCPM-CANN inherits this data efficiency, which amplifies the significance of the retention figures.

### 3. Why the Ascend choice is structurally significant

Until now, the 1.58-bit ecosystem (BitNet b1.58, BitCPM GPU) was exclusively CUDA. All optimized ternary kernel work — integer-accumulation matmul, scaling factor management — relied on NVIDIA primitives. Porting this pipeline to CANN/MindSpeed is non-trivial: it requires rewriting custom operators, handling numerical precision differences across architectures, and validating convergence on a platform with less mature debugging tooling.

The geopolitical subtext is clear: China has a massive installed base of Ascend NPUs (910, 910B, 910C series) in its datacenters, but the LLM software ecosystem remained largely CUDA-dependent through compatibility layers. BitCPM-CANN provides reusable infrastructure — the authors state this explicitly — so other teams can train low-bit models on Ascend without starting from scratch.

### 4. Potential losers and real limitations

**NVIDIA**: every native QAT pipeline that works on Ascend is a demonstration that the CUDA moat on LLM training is narrowing. Not an immediate break, but an industrially serious proof of concept at 8B parameters.

**Post-training quantization vendors**: if native ternary QAT becomes a default configuration with 4.5% overhead, the value proposition of PTQ tooling (llama.cpp Q2_K, AutoGPTQ, etc.) on natively ternary-trained models becomes marginal. The model exits training already quantized.

**Real limitations**: the paper publishes no inference benchmarks — latency, tokens/s — on Ascend with ternary weights. The 8× memory reduction is validated, but actual speedup depends on ternary inference kernel implementation on NPU, which is not documented here. The 0.5B at 90.1% retention remains problematic for edge use cases where that size is precisely the target. Finally, models and code are available on HuggingFace and GitHub, but full reproducibility requires access to Ascend hardware, which constrains independent community validation.