YAML Metadata Warning:The pipeline tag "zero-shot-audio-classification" is not in the official list: text-classification, token-classification, table-question-answering, question-answering, zero-shot-classification, translation, summarization, feature-extraction, text-generation, fill-mask, sentence-similarity, text-to-speech, text-to-audio, automatic-speech-recognition, audio-to-audio, audio-classification, audio-text-to-text, voice-activity-detection, depth-estimation, image-classification, object-detection, image-segmentation, text-to-image, image-to-text, image-to-image, image-to-video, unconditional-image-generation, video-classification, reinforcement-learning, robotics, tabular-classification, tabular-regression, tabular-to-text, table-to-text, multiple-choice, text-ranking, text-retrieval, time-series-forecasting, text-to-video, image-text-to-text, image-text-to-image, image-text-to-video, visual-question-answering, document-question-answering, zero-shot-image-classification, graph-ml, mask-generation, zero-shot-object-detection, text-to-3d, image-to-3d, image-feature-extraction, video-text-to-text, keypoint-detection, visual-document-retrieval, any-to-any, video-to-video, other

larger_clap_general, ONNX, 8-bit

A modified version of laion/larger_clap_general (revision ada0c23a36c4e8582805bb38fec3905903f18b41), © LAION, licensed under Apache-2.0.

Changes from the original

As required by Apache-2.0 §4(b):

Exported to ONNX (opset 18) as two graphs, one for the audio tower and one for the text tower, each with its projection. Both return unit-length 512-number embeddings.
Feature extraction built in. The audio graph takes 32 kHz mono samples and holds a windowed-sinc resampler to 48 kHz and the ClapFeatureExtractor log-mel spectrogram (1024-point Hann frames, hop 480, 64 slaney mels from 50 Hz to 14 kHz, power in dB).
Audio weights quantized to 8 bits. ONNX Runtime quantize_dynamic: dynamic int8 MatMuls. Cosine 0.998 to the float model.
Text weights quantized to 8 bits. ONNX Runtime MatMulNBitsQuantizer (8 bits, block size 32, symmetric) on the MatMul weights, activations kept float, and an int8 token embedding (Gather). Cosine 0.998 to the float model.

The tokenizer is not included; use vocab.json and merges.txt from the original repository. export_clap.py (with common.py) made these files and checks them against transformers.

Files

file	bytes	SHA-256
`larger-clap-general-audio.onnx`	78,682,146	`4e286fc4ba1effb817201af133a5f4ad27fdf22cd824d113a94cb575a1ff6170`
`larger-clap-general-text.onnx`	141,210,922	`51a34ade0421d12b9e8b15754142f8d76db273dc159548cd758ac5902e75a991`

Audio

input	type	shape	meaning
`samples`	f32	`[W, 320000]`	`W` windows of 10 s of mono 32 kHz audio

output	type	shape	meaning
`embedding`	f32	`[W, 512]`	one unit-length vector per window

Shorter clips are repeated to fill the 10 s and padded with zeros, as ClapFeatureExtractor does ("repeatpad").

Text

input	type	shape	meaning
`input_ids`	i64	`[T, L]`	RoBERTa token ids of `T` texts, at most 77 tokens
`attention_mask`	i64	`[T, L]`	1 for tokens, 0 for padding

output	type	shape	meaning
`embedding`	f32	`[T, 512]`	one unit-length vector per text

The dot product of an audio and a text embedding is their similarity.

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Chris-Kode/larger-clap-general-onnx

Base model

laion/larger_clap_general

Quantized

(4)

this model