laion/music-popularity

Music Popularity Predictor

Predicts play count and upvote/like count of AI-generated music tracks from audio alone.

Companion release of the NeurIPS 2026 Evaluations & Datasets Track paper LAION-Tunes: An Open 1.4M-Track Dataset and Perceptual Benchmark for AI-Generated Music (Kaczmarczyk, Ahmed, Friedrich, Erickson, Sharoni, Herremans, Schuhmann). Code: LAION-AI/tunes. Demo: Tunes-Search.

Architecture

Component	Details
Encoder	laion/music-whisper (Whisper Small fine-tuned for music captioning, frozen)
Pooling	Encoder output (1500x768) → 10 segments of 150 frames → mean/max/min pool → 23,040-dim
MLP Head	23040 → 1024 → 256 (LayerNorm) → two prediction heads (play count + upvote count)
Output	log1p-scaled: log(1 + count) — use math.expm1() to convert back

Training

• Data: ~39,000 samples from the broader LAION-Tunes training pool with real platform statistics (Suno, Udio, Riffusion, Sonauto). Mureka exposes no engagement counts and is excluded. Sampling is stratified by log-play quantile x platform, with equal sampling per bucket.
• Loss: Huber loss
• Optimizer: AdamW (lr=5e-4, weight_decay=1e-4, cosine annealing)
• Epochs: 3 (best validation loss 4.004 at epoch 2); batch size 6 on a single A100
• Validation: 200 held-out tracks, stratified by play-count bucket (50 per bucket)

Evaluation (200 validation samples)

The held-out set is stratified by play count, so the log-Pearson r describes how well the head separates low- from high-engagement tracks and need not transfer to a random draw from the corpus.

Metric	Play Count	Upvote Count
Pearson r	0.145	0.102
Log-Pearson r	0.414	0.413
Log MAE	2.981	1.923

Usage

import torch
import torch.nn as nn
import numpy as np
import librosa
import math
from transformers import WhisperProcessor, WhisperForConditionalGeneration
from huggingface_hub import hf_hub_download


# --- Define the MLP head ---

class PopularityMLP(nn.Module):
    def __init__(self):
        super().__init__()
        self.bottleneck = nn.Sequential(
            nn.Linear(23040, 1024), nn.ReLU(), nn.Dropout(0.3),
            nn.Linear(1024, 256), nn.ReLU(), nn.LayerNorm(256),
        )
        self.play_head = nn.Sequential(nn.Linear(256, 64), nn.ReLU(), nn.Linear(64, 1))
        self.upvote_head = nn.Sequential(nn.Linear(256, 64), nn.ReLU(), nn.Linear(64, 1))

    def forward(self, x):
        feat = self.bottleneck(x)
        return self.play_head(feat).squeeze(-1), self.upvote_head(feat).squeeze(-1)


# --- Load models ---

# Whisper encoder from laion/music-whisper
processor = WhisperProcessor.from_pretrained("laion/music-whisper")
whisper = WhisperForConditionalGeneration.from_pretrained(
    "laion/music-whisper", torch_dtype=torch.float16
).cuda().eval()
encoder = whisper.get_encoder()

# Popularity head from this repo
head_path = hf_hub_download("laion/music-popularity", "popularity_head.pt")
mlp = PopularityMLP().cuda()
mlp.load_state_dict(torch.load(head_path, map_location="cuda")["mlp_state_dict"])
mlp.eval()


# --- Run inference ---

audio, sr = librosa.load("song.mp3", sr=16000, mono=True)
audio = audio[:30 * 16000]  # first 30 seconds
if len(audio) < 30 * 16000:
    audio = np.pad(audio, (0, 30 * 16000 - len(audio)))

inputs = processor(audio, sampling_rate=16000, return_tensors="pt")

with torch.no_grad():
    enc_out = encoder(inputs.input_features.cuda().half()).last_hidden_state  # (1, 1500, 768)

    # Segment pooling: 10 segments, mean/max/min
    segments = enc_out.view(1, 10, 150, 768)
    pooled = torch.cat([segments.mean(2), segments.max(2).values, segments.min(2).values], dim=2)
    pooled = pooled.view(1, -1).float()  # (1, 23040)

    pred_play, pred_upvote = mlp(pooled)

print(f"Estimated plays:   {math.expm1(pred_play.item()):,.0f}")
print(f"Estimated upvotes: {math.expm1(pred_upvote.item()):,.0f}")

Files

File	Description
popularity_head.pt	MLP head weights (91 MB)
evaluation_report.html	Detailed evaluation with plots

The Whisper encoder is loaded separately from laion/music-whisper.

License

Apache 2.0 — Christoph Schuhmann / LAION

Acknowledgments

• Encoder: laion/music-whisper (OpenAI Whisper Small, fine-tuned for music captioning)
• Data: the LAION-Tunes training pool (Suno, Udio, Riffusion, Sonauto); released corpus: laion/laion-tunes
• Developed by Christoph Schuhmann and the LAION community

Citation

@inproceedings{kaczmarczyk2026laiontunes,
  title     = {{LAION-Tunes}: An Open 1.4{M}-Track Dataset and Perceptual Benchmark for {AI}-Generated Music},
  author    = {Kaczmarczyk, Robert and Ahmed, Tawsif and Friedrich, Felix and Erickson, Aidan C. and Sharoni, Orian and Herremans, Dorien and Schuhmann, Christoph},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS), Evaluations and Datasets Track},
  year      = {2026}
}