Kimi-K2.5 ยท REAP (192/384 experts) + MoE-SQ paired 4:8 NVFP4

Rebuttal checkpoint for Hardware-Native Joint Sparse-Quantization for Trillion-Scale Mixture-of-Experts (MoE-SQ, arXiv 2610.02241): a matched-bit-budget comparison of expert pruning (REAP) and MoE-SQ.

routed experts 192 of 384 per layer, top-8 routing unchanged
bits per routed-expert weight 1.375 (counted against all 384 experts' weights, scales and metadata included)
size ~199 GB
left in bf16 attention, shared expert, dense first layer, router, embeddings, lm_head, vision tower

How it was made

A slice of ISTA-DASLab/Kimi-K2.5-P48NVFP4-MoESQ: the REAP-kept experts' tensors are copied byte-for-byte (paired-4:8 sparse storage), renumbered in order, and the router rows / score-correction bias are sliced; 192 of 384 experts removed per layer. Nothing is recompressed, so the MoE-SQ compression never saw the pruned model. Kept indices from REAP (llm-compressor 0.13.0, mixed calibration, 512 ร— 4096) are in reap_kept_experts.jsonl.

Notes:

  • Stage 1 and the slicing ran on 8ร— NVIDIA B300 (SM103).

Serving

Same stack as the other MoE-SQ / REAP Kimi checkpoints: vLLM v0.30.0 with the MoE-SQ patch (IST-DASLab/MoESQ, integrations/vllm/install.sh). Multimodal paths are untested; all evaluations are text-only.

License and attribution

Base model moonshotai/Kimi-K2.5 under a Modified MIT License (see LICENSE, which applies to this checkpoint); upstream notices in THIRD_PARTY_NOTICES.md. REAP is due to Cerebras Research; the implementation used is llm-compressor's REAPPruningModifier.

Downloads last month
36
Safetensors
Model size
329B params
Tensor type
BF16
ยท
U8
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for kwanhee/Kimi-K2.5-REAP-E192-P48NVFP4-MoESQ

Quantized
(44)
this model