--- license: apache-2.0 language: - en base_model: - Axion1-0/Axion-1.0-35B pipeline_tag: text-generation library_name: transformers tags: - agent - tool-use - scientific-reasoning - qwen3_5_moe - MXFP4 - moe - text-generation-inference - llama-cpp - nvfp4 - nvidia - llama-cli - nvidia-blackwell - rtx-pro-6000 - SM120 - GB202 - B200 - B300 - FP4 - blackwell - qwen3_5 - Qwen3.5 - Qwen3.6 - Qwen3.8 - 35B - A3B - reproduction datasets: - Axion1-0/axion-training-data --- # **Axion-1.0-35B-GGUF** > Axion 1.0-35B is an Apache-2.0 reproduction checkpoint of ByteDance-linked Axion 1.0, following the recipe in *Axion 1.0: Open Agent Foundation Models, Simplified*. The 35B MoE model has approximately 3B active parameters and targets scientific reasoning, information seeking, and tool-augmented problem solving. Its three-stage training combines roughly 18K SFT examples for reasoning and investigation with reinforcement learning over agent trajectories. Benchmarks include HLE 52.8, BMB 35.3, SEAL-0 60.4, HiPhO 45.9, FrontierScience Olympiad 85.0, and FrontierScience Research 55.0, closely tracking the paper’s reported results. It leads the comparison table on SEAL-0 and performs strongly on HiPhO and FrontierScience Olympiad, while trailing top models on HLE, BMB, and FrontierScience Research. Released in BF16, it supports vLLM serving through an OpenAI-compatible endpoint; tool-use evaluation requires a separate execution harness. > [!NOTE] > MTP — Speculative Decoding: No | Don't know which one to choose? Grab MXFP4_MOE (best for NVIDIA Blackwell) or Q4_K_M for balanced performance. ## Model Files | File Name | Quant Type | File Size | File Link | Description | |-----------|------------|-----------|-----------|-------------| | Axion-1.0-35B.BF16.gguf | BF16 | 69.4 GB | [Link](https://huggingface.co/prithivMLmods/Axion-1.0-35B-GGUF/blob/main/Axion-1.0-35B.BF16.gguf) | Full BF16 weights. Highest quality, largest file size. | | Axion-1.0-35B.MXFP4_MOE.gguf | MXFP4_MOE | 19.8 GB | [Link](https://huggingface.co/prithivMLmods/Axion-1.0-35B-GGUF/blob/main/Axion-1.0-35B.MXFP4_MOE.gguf) | **MXFP4_MOE**: Highly optimized for **NVIDIA Blackwell** hardware accelerators **(e.g., RTX Pro 6000, RTX 50-series, B200, B300)**, offering native hardware-accelerated FP4 Tensor Core speeds and a strong balance of size and performance. | | Axion-1.0-35B.Q3_K_L.gguf | Q3_K_L | 18.1 GB | [Link](https://huggingface.co/prithivMLmods/Axion-1.0-35B-GGUF/blob/main/Axion-1.0-35B.Q3_K_L.gguf) | Lower quality but usable, good for low RAM availability. | | Axion-1.0-35B.Q3_K_M.gguf | Q3_K_M | 16.8 GB | [Link](https://huggingface.co/prithivMLmods/Axion-1.0-35B-GGUF/blob/main/Axion-1.0-35B.Q3_K_M.gguf) | Low quality. | | Axion-1.0-35B.Q4_K_M.gguf | Q4_K_M | 21.2 GB | [Link](https://huggingface.co/prithivMLmods/Axion-1.0-35B-GGUF/blob/main/Axion-1.0-35B.Q4_K_M.gguf) | Good quality, default size for most use cases, *recommended*. | | Axion-1.0-35B.Q5_K_M.gguf | Q5_K_M | 24.7 GB | [Link](https://huggingface.co/prithivMLmods/Axion-1.0-35B-GGUF/blob/main/Axion-1.0-35B.Q5_K_M.gguf) | High quality, *recommended* for standard hardware. | | Axion-1.0-35B.Q6_K.gguf | Q6_K | 28.5 GB | [Link](https://huggingface.co/prithivMLmods/Axion-1.0-35B-GGUF/blob/main/Axion-1.0-35B.Q6_K.gguf) | Very high quality, near perfect, *recommended* for standard hardware. | | Axion-1.0-35B.mmproj-bf16.gguf | mmproj-bf16 | 903 MB | [Link](https://huggingface.co/prithivMLmods/Axion-1.0-35B-GGUF/blob/main/Axion-1.0-35B.mmproj-bf16.gguf) | Multimodal projection file in BF16 format. Used for vision/language models. | ## MXFP4_MOE Test Run Tested `Axion-1.0-35B.MXFP4_MOE.gguf` on an NVIDIA RTX PRO 6000 Blackwell Server Edition GPU with `96 GB` of VRAM, using `llama-cli` with full GPU offloading. ### Performance Metrics | Metric | Value | | --- | --- | | Average Prompt Processing Speed | 238.50 tokens/s | | Average Generation Speed | 211.70 tokens/s | | Total Wall Time | 86.1 seconds | | Context Length | 16,384 tokens | | KV Cache (K) | `q8_0` | | GPU | NVIDIA RTX PRO 6000 Blackwell Server Edition | | GPU VRAM Capacity | 96 GB | | Peak GPU VRAM Usage | 19.07 GiB | | Peak GPU Utilization | 87% | | GPU Layers Offloaded | 99 (`-ngl 99`) | | Random Seed | 42 | | Speculative Decoding | No | | Test Runs | 5 | ### Test Shots | Shot | Test | | --- | --- | | Shot 1 | C: Insert Node at End of Singly Linked List | | Shot 2 | Jogger and Train Passing Time | | Shot 3 | Two Pipes Filling a Tank | | Shot 4 | Students in Examination Rooms | | Shot 5 | Father and Son Ages | ### Run Inference ```bash ./build/bin/llama-cli \ -m /path/to/models/Axion-1.0-35B.MXFP4_MOE.gguf \ -ngl 99 \ -c 16384 \ --cache-type-k q8_0 \ --seed 42 \ -n 8192 \ -st \ -p "Your prompt here" ``` *Performance results reflect this specific hardware and inference configuration. Actual throughput may vary depending on the prompt, model, software build, and runtime settings.* ## Snapshots (5) ![download (1)](https://cdn-uploads.huggingface.co/production/uploads/65bb837dbfb878f46c77de4c/ir6GijyS5_bvdKCrKXA__.png) ![download (2)](https://cdn-uploads.huggingface.co/production/uploads/65bb837dbfb878f46c77de4c/aogF187zuXtwWgZYv_EPk.png) ![download (3)](https://cdn-uploads.huggingface.co/production/uploads/65bb837dbfb878f46c77de4c/8H7-FYNKAfwtzwHHskX0S.png) ![download (4)](https://cdn-uploads.huggingface.co/production/uploads/65bb837dbfb878f46c77de4c/z_-z-05UUpEbUzMZpn0fj.png) ![download (5)](https://cdn-uploads.huggingface.co/production/uploads/65bb837dbfb878f46c77de4c/BBzK72JPKnL4U3I7HjJCM.png) ![download](https://cdn-uploads.huggingface.co/production/uploads/65bb837dbfb878f46c77de4c/9P1nsKfsvTRFRGyJzswRI.png) ## llama.cpp LLM inference in C/C++ — [https://github.com/ggml-org/llama.cpp](https://github.com/ggml-org/llama.cpp) Quantized using llama.cpp by [prithivMLmods](https://huggingface.co/prithivMLmods)