Qwen3.5-9B-Claude-Opus-4.7-GGUF

Description

This repo contains GGUF weights for the fine-tuned Qwen3.5-9B-Claude-Opus-4.7. This is a 9-billion parameter model distilled from advanced reasoning chains generated by Claude Opus 4.7, optimized for high-level logical deduction and step-by-step problem solving.

As a Vision-Language-Model (VLM), it retains the base model's ability to analyze images while applying the enhanced reasoning logic learned during the fine-tuning process.

Available Files

For the best balance of performance and intelligence, we recommend the Q4_K_M or Q5_K_M versions.

• Qwen3.5-9B.Q4_K_M.gguf: Recommended for most users (4-bit quantization).
• Qwen3.5-9B.Q5_K_M.gguf: High fidelity (5-bit quantization).
• Qwen3.5-9B.Q8_0.gguf: Near-perfect precision (8-bit quantization).
• Qwen3.5-9B.F16.gguf: Full weights (16-bit).
• Qwen3.5-9B.BF16-mmproj.gguf: Required for Vision functionality. Use this alongside any of the GGUF files above to enable image analysis.

---

Usage Instructions

1. Multimodal (Vision + Text)

To use the model with images, you must provide the multimodal projector (mmproj) file:

llama-mtmd-cli -hf keypa/Qwen3.5-9B-Claude-Opus-4.7-GGUF --mmproj Qwen3.5-9B.BF16-mmproj.gguf --jinja

2. Text-Only Reasoning

For standard logic and chat tasks:

llama-cli -hf keypa/Qwen3.5-9B-Claude-Opus-4.7-GGUF --jinja

3. Prompting for Reasoning

This model was trained to think before it speaks. For best results, use the following template structure:

<|im_start|>system
You are a helpful assistant with advanced reasoning capabilities.<|im_end|>
<|im_start|>user
[Your Question or Image Here]<|im_end|>
<|im_start|>assistant
<|im_thought|>

---

Technical Details

• Finetuned from: Qwen/Qwen3.5-9B
• Training Method: QLoRA via Unsloth
• Dataset:lordx64/reasoning-distill-claude-opus-4-7-max (Distilled from Claude Opus 4.7)
• Conversion: Merged to 16-bit and quantized via llama.cpp

This model was trained 2x faster with Unsloth.