Instructions to use hitheshkaranth/Skanda-0-int8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sam2
How to use hitheshkaranth/Skanda-0-int8 with sam2:
# Use SAM2 with images import torch from sam2.sam2_image_predictor import SAM2ImagePredictor predictor = SAM2ImagePredictor.from_pretrained("hitheshkaranth/Skanda-0-int8") with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16): predictor.set_image(<your_image>) masks, _, _ = predictor.predict(<input_prompts>)# Use SAM2 with videos import torch from sam2.sam2_video_predictor import SAM2VideoPredictor predictor = SAM2VideoPredictor.from_pretrained("hitheshkaranth/Skanda-0-int8") with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16): state = predictor.init_state(<your_video>) # add new prompts and instantly get the output on the same frame frame_idx, object_ids, masks = predictor.add_new_points(state, <your_prompts>) # propagate the prompts to get masklets throughout the video for frame_idx, object_ids, masks in predictor.propagate_in_video(state): ... - Notebooks
- Google Colab
- Kaggle
Research use only. Skanda drafts findings for a radiologist to review. It is not a medical device and has not been cleared or approved for clinical use by any regulator.
Skanda reads chest X-ray, CT and MRI, writes a structured report and outlines the findings it reports:
Qwen3-VL-30B-A3B with a [SEG] token per finding decoded by SAM 2.1. Limitations (rare X-ray findings, missed findings,
CT outlines on vessels, small internal test sets) are listed on the Skanda-0 card. This repo holds the int8 build of Skanda-0 (its main checkpoint, run 3):
32.8 GB instead of 62.6 GB.
Only the language model's weights are quantized. The SAM 2.1 decoder, the [SEG] projector and the score head stay
in bf16. The abnormal gate was re-calibrated on this build's own reads of the calibration set (validation data,
never test).
int8 is effectively lossless. Loss per token on the bf16 model's own replies is 0.0824 vs 0.0823, and every benchmark number stays within about a point of bf16, except lesion-free CT cleared (91% β 85%). Recommended build.
π Benchmark: Skanda vs MedGemma 27B, MAIRA-2 and MedGemma + MAIRA-2, each at bf16, 8-bit and 4-bit
Percent on held-out test cases (no test patient was seen in training):
- Chest X-ray: 111 ChestX-Det films, scored by the Skanda Radiology app's own benchmark; mean of two passes.
- CT / MRI: a fixed held-out set of slices and volumes, with and without lesions.
The bf16 Skanda column was re-read on 8 Nov 2026 with the same code and calibration steps as the quantized builds. Its CT/MRI reads were identical to the original 1 Nov reads (580 of 580).
MedGemma + MAIRA-2 is the Skanda Radiology app's fused mode at the matching precision. MAIRA-2 reads chest X-rays only. The baselines are evaluated here only; their licences are their own (MAIRA-2: research use only).
Methods
| 8-bit | 4-bit | |
|---|---|---|
| MedGemma 27B | FP8 (vLLM) | MXFP4 (vLLM) |
| MAIRA-2 | int8 (bitsandbytes) | NF4 (bitsandbytes) |
| MedGemma + MAIRA-2 | FP8 + int8 | MXFP4 + NF4 |
| Skanda | int8 per-row (skanda/quant.py) | int4 group-128 (skanda/quant.py) |
| π©» Chest X-ray (111) | MedGemma 27B bf16 | MedGemma 27B 8-bit | MedGemma 27B 4-bit | MAIRA-2 bf16 | MAIRA-2 8-bit | MAIRA-2 4-bit | MedGemma + MAIRA-2 bf16 | MedGemma + MAIRA-2 8-bit | MedGemma + MAIRA-2 4-bit | Skanda bf16 | Skanda 8-bit | Skanda 4-bit |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Abnormal flagged | 86 | 87 | 89 | 89 | 89 | 89 | 87.5 | 88 | 90 | 84.5 | 86 | 85.5 |
| Normals called normal | 72 | 72 | 69 | 72 | 76 | 62 | 72 | 72 | 69 | 69 | 69 | 72 |
| Right finding type | 36 | 38.5 | 34.5 | 34 | 34 | 34 | 42 | 45 | 42 | 39 | 40.5 | 36 |
| Box precision | 8.5 | 8 | 7.5 | 30 | 28 | 28 | 10.5 | 8 | 9 | 24.5 | 24 | 17.5 |
| Box recall | 5.5 | 5.5 | 4 | 13 | 11 | 12 | 7 | 6.5 | 6 | 9.5 | 8.5 | 8.5 |
| π« CT | MedGemma 27B bf16 | MedGemma 27B 8-bit | MedGemma 27B 4-bit | Skanda bf16 | Skanda 8-bit | Skanda 4-bit |
|---|---|---|---|---|---|---|
| Abnormal flagged | 97 | 97 | 96 | 82 | 82 | 82 |
| Normals called normal | 6 | 7 | 8 | 91 | 85 | 78 |
| Right finding type | 49 | 46 | 49 | 72 | 72 | 74 |
| Box precision | 1 | 1 | 1 | 22 | 21 | 19 |
| Box recall | 3 | 3 | 2 | 16 | 16 | 15 |
| π§ MRI | MedGemma 27B bf16 | MedGemma 27B 8-bit | MedGemma 27B 4-bit | Skanda bf16 | Skanda 8-bit | Skanda 4-bit |
|---|---|---|---|---|---|---|
| Abnormal flagged | 99 | 100 | 99 | 89 | 89 | 88 |
| Normals called normal | 2 | 4 | 4 | 94 | 92 | 93 |
| Right finding type | 42 | 46 | 47 | 89 | 87 | 80 |
| Box precision | 6 | 6 | 8 | 70 | 70 | 75 |
| Box recall | 8 | 8 | 8 | 65 | 67 | 49 |
| Loss: NLL per token of own bf16 replies | bf16 | 8-bit | 4-bit |
|---|---|---|---|
| MedGemma 27B | 0.1169 | 0.1224 | 0.1629 |
| MAIRA-2 | 0.2971 | 0.3026 | 0.3456 |
| Skanda | 0.0823 | 0.0824 | 0.1181 |
| Skanda validation NLL (radiologist targets) | 0.1600 | 0.1607 | 0.1865 |
Skanda checkpoint size (GB): bf16 62.6, 8-bit 32.8, 4-bit 18.2
Box precision / recall: X-ray, a predicted box counts when its centre lies inside a radiologist box of the same class; CT/MRI, box IoU β₯ 0.3 with a radiologist outline, matched one-to-one. Each X-ray number moves about Β±10 points by chance across 111 films.
π¦ Files and loading
| File | What it is |
|---|---|
vlm/ |
Qwen3-VL-30B-A3B with the run-3 LoRA merged, quantized (int8 per-row weight-only (MoE experts + attention)); vlm/skanda_quant.json describes the format |
skanda_heads.pt |
SAM 2.1 (image encoder and trained mask decoder), the [SEG] projector and the score head (bf16) |
skanda_config.json |
[SEG] token id, gate thresholds re-calibrated for this build, hint thresholds |
hint_scores.jsonl |
Score-head outputs the hint thresholds were chosen on (validation and dev only) |
Loading needs Skanda's inference code (skanda/serve.py, skanda/quant.py), which is open source (Apache-2.0)
at github.com/Hitheshkaranth/Skanda-0, as for the bf16 checkpoint. skanda/quant.py keeps the weights in
int8 in memory and dequantizes each MoE expert only when a token is routed to it.
huggingface-cli download hitheshkaranth/Skanda-0-int8 --local-dir Skanda-0-int8
# in a checkout of github.com/Hitheshkaranth/Skanda-0
from PIL import Image
from skanda.serve import Skanda
sk = Skanda("Skanda-0-int8") # the bf16, int8 and int4 builds load the same way
report, raw_text, finished, n_tokens = sk.read(Image.open("chest.png").convert("RGB"))
πΎ Memory
| Build | Download | Memory, weights | Memory, peak | Seconds per read |
|---|---|---|---|---|
| bf16 | 62.6 GB | 62.6 GB | 63.9 GB | 7.0 |
| int8 (this repo) | 32.8 GB | 32.7 GB | 34.7 GB | 16.5 |
| int4 | 18.2 GB | 18.2 GB | 20.1 GB | 23.0 |
Measured with PyTorch on an NVIDIA DGX Spark (GB10, 128 GB unified memory) on builds of a later Skanda checkpoint with the same architecture and quantization code. Download sizes are this checkpoint's.
π Licence
CC BY-NC 4.0, like Skanda-0: research and other non-commercial use
with attribution. Commercial use requires written permission from the author (request it in the
Skanda-0 Discussions). Licence text in LICENSE;
upstream licences and data attributions in NOTICE. Research use only; not a medical device.
π§ Architecture and training
Same model as Skanda-0: Qwen3-VL-30B-A3B writes the report, the hidden state at each [SEG] token goes through a projector into SAM 2.1's mask decoder, and a score head gives p(abnormal) and 13 X-ray classes. Only the language model's MoE experts and attention are stored in int8; the vision encoder, router, [SEG] rows, projector, score head and SAM 2.1 stay bf16. Architecture diagram, losses and the exact six-pass training recipe: Skanda-0 card and github.com/Hitheshkaranth/Skanda-0.
Other builds of Skanda-0
- bf16:
hitheshkaranth/Skanda-0 - int4:
hitheshkaranth/Skanda-0-int4
- Downloads last month
- 70