SmolVLM2-2.2B — LiteRT-LM (on-device Vision-Language Model)

HuggingFaceTB/SmolVLM2-2.2B-Instruct (image path) converted to the LiteRT-LM (.litertlm) format for on-device image+text inference with Google's LiteRT-LM runtime.

SmolVLM2-2.2B is the largest / most capable of Hugging Face's SmolVLM2 family: a SigLIP vision encoder + pixel-shuffle connector feeding a SmolLM2-1.7B (Llama-architecture) language decoder. Give it an image and a question, get a grounded answer, fully offline.

File	SmolVLM2-2.2B.litertlm
Vision	SigLIP encoder (384×384, patch 14 → 729 patches, no CLS) + pixel-shuffle ×3 + Linear connector, int8 → 81 image tokens
Decoder	SmolLM2-1.7B (Llama, 2048-dim, 24 layers), int4 weights (blockwise-32 + OCTAV); tied embedding INT8 (externalized)
Compute	integer
Context (KV cache)	2048
Image input	resized to 384×384 ((x−0.5)/0.5 normalization baked into the vision encoder)
Base model	HuggingFaceTB/SmolVLM2-2.2B-Instruct

How to use

1. Install the runtime

pip install litert-lm

2. Run it in one command — this downloads the bundle, encodes your image and answers:

litert-lm run --from-huggingface-repo litert-community/SmolVLM2-2.2B SmolVLM2-2.2B.litertlm \
  --attachment photo.jpg \
  --prompt "Describe this image in one sentence."

On the COCO sample image huggingface/documentation-images/coco_sample.png (two tabby cats on a pink blanket, remote controls beside them) this prints:

Two cats are sleeping on a pink blanket with remote controls nearby.

Drop --prompt for an interactive chat, and pass --attachment more than once for several images. litert-lm serve exposes the same bundle as a local OpenAI-compatible API. The same file runs on macOS, Linux and Windows.

Performance

litert-lm benchmark (litert-lm 0.15.0) on an Apple M4 Max, -p 256 -d 256 --runs 3 (the tool averages three iterations), max-num-tokens 4096, warm-up run discarded, otherwise idle machine. These figures cover the text path; the vision encoder runs once per image and is not included.

Device	Backend	Prefill (256)	Decode	TTFT
Apple M4 Max (macOS)	CPU	119 tok/s	33.6 tok/s	2.34 s
Apple M4 Max (macOS)	GPU (Metal)	1608 tok/s	134.7 tok/s	0.18 s

Every desktop backend listed above was checked by actually generating a caption on it, not just by reading the benchmark tool's output.

Accuracy note

Single-image VQA produces coherent, image-grounded answers (the SigLIP vision tower converts bit-faithfully to the reference, float CPU-parity corr ≈ 1.0). This is the largest SmolVLM2, so it is notably more capable than SmolVLM2-500M.

Galaxy S26 — GPU backend

The published bundle runs on the Android GPU backend and generates.

file	GPU backend	delegation	peak
SmolVLM2-2.2B.litertlm	runs	2979 / 2979 ops across 3 subgraphs on LiteRT GPU	863 MB

Measured on a Samsung Galaxy S26 (SM-S942Q / SM8850, Android 16) with litert_lm_advanced_main from litert-lm 0.16.0, --backend=gpu --sampler_backend=cpu, prompt What is the capital of France?. Peak is the process high-water mark (VmHWM) sampled during that same run. Gated 2026-08-25.

The op counts above are the LiteRT GPU partitions. XNNPACK additionally takes 1 of the 4 nodes in main; the runtime accepts that split.

The gate prompt carries no image, so this covers engine creation and the text path. The vision path on the GPU is not measured here.

No speed rows, on purpose. On this handset the GPU backend wins prefill and does not win decode, so a GPU throughput figure only means something beside a CPU row from the same handset, and no S26 CPU row exists for this model yet.

GPU wiring, including the Gallery import toggle: GPU guide.

⚠️ Best for single-image VQA — one image per conversation

Ask about one image per chat (start a new conversation for a different image).

Run on Android — Google AI Edge Gallery

Update (July 2026): Google AI Edge Gallery v1.0.16+ can import litert-lm models directly from Hugging Face inside the app (tap +) — no computer or adb needed. The manual steps below are only required on older builds or for sideloading a local file.

Run this model with image input in the official Google AI Edge Gallery app — no custom app needed:

1. Push the bundle onto the phone (or download it there directly from this repo): adb push SmolVLM2-2.2B.litertlm /sdcard/Download/
2. Open the Gallery app, tap the + icon (bottom-right) and pick SmolVLM2-2.2B.litertlm.
3. In the Import Model dialog, check "Support image" (required for image input), then tap Import.
4. Open the Ask Image task, choose the imported model, attach a photo, and ask.

Run on desktop (LiteRT-LM CLI)

The same .litertlm bundle runs on macOS / Linux / Windows with the official LiteRT-LM CLI — including as a local OpenAI-compatible API server:

pip install litert-lm
litert-lm import --from-huggingface-repo litert-community/SmolVLM2-2.2B SmolVLM2-2.2B.litertlm smolvlm2-2.2b
litert-lm run smolvlm2-2.2b     # interactive chat in the terminal
litert-lm serve           # local OpenAI-compatible API server

Run on iPhone / macOS

Use the LiteRT-LM Swift runtime (swift-litert-lm / the LiteRTDemo sample). Load SmolVLM2-2.2B.litertlm with the vision tower enabled (modalities Modality.textImage / [.vision]), attach a photo, and ask.

Conversion notes

• LiteRT-LM fast_vlm bundle: VISION_ENCODER ([1,384,384,3]→SigLIP) + VISION_ADAPTER (pixel-shuffle ×3 + Linear → [1,81,2048]) + single-token EMBEDDER + PREFILL_DECODE.
• The vision encoder uses the static position-embedding path (the model's dynamic bucketize position logic is bypassed — numerically identical for a full 384×384 frame) and bakes the (x−0.5)/0.5 normalization + NCHW transpose into the graph.
• Single-image, no high-res splitting → a fixed 81 soft tokens; SmolLM2 (Llama) decoder exported with externalized (tied) embedder.

2026-08-28 — start_token fix (weights unchanged)

The LiteRT-LM engine prepends the metadata start_token to every prompt, and this model's chat template already renders <|im_start|> itself — so the model was reading <|im_start|><|im_start|>…, a stream it was never trained on. The start token has been removed.

Metadata-only change: every section of the bundle except the LlmMetadata block is byte-identical to the previous file (verified by sha256 per section), so the weights, the graph and the tokenizer are unchanged and the speed and memory numbers on this card still describe exactly this file — only the file's own sha256 differs. What changed is the input: the token stream the model reads for a given conversation can differ from the previous file's, and it now matches this model's own reference chat stream. Greedy decoding can turn on a single token, so an individual answer can differ from the previous file in either direction. Unless a row says otherwise, the accuracy figures on this card were measured on the previous file and have not been re-measured on this one. If you downloaded before 2026-08-28, re-download.

2026-08-30 — tokenizer section replaced (weights unchanged)

The tokenizer in SmolVLM2-2.2B.litertlm was a SentencePiece conversion of the model's BPE tokenizer, and the conversion lost the byte-level semantics: a standalone accented letter or symbol (é, ñ, ü, °, ·, …) was encoded to the id of a single-byte token instead of the token the upstream tokenizer uses, and any character without a whole-character vocabulary entry (emoji, most of Latin Extended-A) became the token the conversion had reused as UNK — the end-of-text token for this vocabulary. SmolVLM2-2.2B.litertlm now embeds the upstream tokenizer.json (the same HF tokenizer path most bundles in this collection use).

Tokenizer-only change: every section of the bundle except the tokenizer is byte-identical to the previous file (verified by sha256 per section), so the weights, the graph and the chat template are unchanged and the speed and memory numbers on this card still describe this file per token — only the file's own sha256 differs. Verified on the LiteRT-LM runtime: the default turn, 7 probe strings, the 223 standalone characters U+00A1–U+017F and every special token now tokenize identically to the upstream tokenizer, and the four ASCII-only test questions answer byte-identically to the previous file (same ids in, same tokens out). Prompts containing accented letters, symbols or emoji reach the model differently from before, so individual answers to such prompts can change. Unless a row says otherwise, the accuracy figures on this card were measured on the previous file, and any on-device rows were measured on the previous file too — the on-device gate has not been re-run on this one (the runtime's tokenizer code is the same on macOS and on device; the weights and graph are byte-identical). If you downloaded before 2026-08-30, re-download.