zhangsongbo365/Qwen3-VL-Embedding-2B-onnx

English | 中文

Qwen3-VL-Embedding-2B — ONNX (16-frame & 32-frame video variants)

ONNX exports of Qwen/Qwen3-VL-Embedding-2B for video embedding at two temporal window sizes, split into a vision encoder and a transformer decoder. Based on the PIA-SPACE-LAB/Qwen3-VL-Embedding-2B-onnx export pipeline, with two export bugs fixed (see below) and both variants precision-verified end-to-end against PyTorch.

Variants

Each folder is self-contained (ONNX + rotary params + tokenizer).

Folder	Raw frames	TEMPORAL_SIZE	Temporal patches	Vision input	Vision tokens	Verified
temporal16/	16	16	8	[18432, 1536]	4608	✅ embedding cos 1.000000
temporal32/	32	32	16	[36864, 1536]	9216	✅ embedding cos 1.000000

Both bake IMG_SIZE = 768×768 per frame (must be a multiple of patch_size × merge_size = 32). Only the transformer's seq_len is dynamic.

Semantics note: in this pipeline TEMPORAL_SIZE counts raw frames, not temporal patches — temporal_patches = ceil(TEMPORAL_SIZE / temporal_patch_size) with temporal_patch_size = 2.

Fixes vs the upstream export

1. Vision attention head/seq layout — the chunked-attention wrapper used for memory-feasible long-sequence export missed the per-split transpose(1, 2), so head and sequence axes were mixed by the final reshape. All vision outputs were permuted; fixed and verified.
2. Text rotate_half convention — the decoder used a plain flip [x2|x1] instead of HF's [-x2|x1]. Verified against HF apply_rotary_pos_emb (diff = 0 after fix). Standard sin = cat(s, s) from the HF rotary output must be used.

Precision validation

PyTorch reference: Qwen3VLForEmbedding, fp32, CPU, real video (hospital_pudong_000.mp4), 768×768 per frame.

Quantity	temporal16 (seq 4692)	temporal32 (seq 9370)
deepstack_feature_0/1/2	cos 1.000000	cos 1.000000
vision_hidden_states	cos 1.000000	cos 1.000000
last_hidden_state (all tokens)	cos 0.999994	cos 0.999931
Pooled embedding (last token, L2-normed)	cos 1.000000	cos 1.000000

verify_precision.py contains the full reference implementation (frame sampling, preprocessing, interleaved-MRoPE construction, ONNX glue) and doubles as inference reference code.

Repository contents

├── temporal16/            16-frame variant (self-contained)
│   ├── Vision.onnx (+ .data)
│   ├── Transformer.onnx (+ .data)
│   ├── rotary_params.npz
│   └── tokenizer/
├── temporal32/            32-frame variant (self-contained)
│   └── … (same layout)
├── export_script/         fixed export scripts (TEMPORAL_SIZE via env var, default 32)
├── verify_precision.py    PyTorch-vs-ONNX precision check + reference inference glue
└── README.md

rotary_params.npz per variant: mRoPE parameters (inv_freq, mrope_section), token-embedding table embed_weight [151936, 2048], grid config (temporal_size, temporal_patches, image_height/width, …). Required at inference.

Usage

1. Sample 16 (temporal16) or 32 (temporal32) frames, resize each to 768×768, preprocess with that folder's tokenizer/ (do_resize=False) → pixel_values_videos / video_grid_thw = [t/2, 48, 48].
2. Run Vision.onnx → 3 deepstack features + vision_hidden_states ([4608|9216, 2048]).
3. Look up text token embeddings from rotary_params.npz:embed_weight; replace video-token positions (video_token_id = 151656) with vision_hidden_states.
4. Build interleaved MRoPE cos/sin (mrope_section = [24, 20, 20], T/H/W interleaved every 3rd frequency, T filling the remainder — matching HF Qwen3VLTextRotaryEmbedding) from inv_freq and per-token 3D position ids (Qwen3VLModel.get_rope_index), shaped [1, seq, 1, 1, 128].
5. Scatter deepstack features into full-sequence tensors (zeros at text positions) — the decoder graph adds them over the whole sequence after layers 0/1/2.
6. Run Transformer.onnx with a causal mask (0 / float32 min) → last_hidden_state; pool the last token and L2-normalize.

See verify_precision.py for a complete working implementation (set N_FRAMES = 16 or 32).

Notes

• Tokenization: sequences need ~4700 tokens (16-frame) or ~9400 tokens (32-frame) — set the processor max_length accordingly (e.g. 8192 / 16384); the upstream default of 8192 truncates 32-frame video tokens.
• Memory: attention peaks are bounded by exact query/head chunking baked into the graphs — both variants run comfortably in ~16 GB RAM with ONNX Runtime CPU.
• Frames attend across the whole temporal window (full attention over all vision patches), matching the HF model's behaviour.
• License Apache-2.0, inherited from the base model. See Qwen/Qwen3-VL-Embedding-2B for upstream terms.