Encoder comparison after equivalent retrieval fine-tuning
ModernBERT, Ettin and UltraBERT: 54 retrieval models, frozen data and results. Validation-selected models first. Matched training, one seed.
38.2 MBNote All 18 completed comparisons, plots, per-query rankings, all LR trial scores, logs and source scripts.
tomaarsen/encoder-comparison-20261014
Updated • 6.3kNote Frozen inputs: 32,768 MS MARCO triplets, 1,024 validation queries, and NanoNQ, NanoFiQA2018 and NanoSciFact test data.
tomaarsen/modernbert-base-dense-msmarco32k-1e-4-20261014
Sentence Similarity • 0.1B • Updated • 41Note NanoBEIR@3 nDCG@10 = 0.5021
tomaarsen/modernbert-base-multivector-msmarco32k-3e-4-20261014
Feature Extraction • 0.1B • Updated • 38Note NanoBEIR@3 nDCG@10 = 0.5526
tomaarsen/modernbert-base-reranker-msmarco32k-1e-4-20261014
Text Ranking • 0.1B • Updated • 57Note NanoBEIR@3 nDCG@10 = 0.5653
tomaarsen/ettin-encoder-150m-dense-msmarco32k-1e-4-20261014
Sentence Similarity • 0.1B • Updated • 76Note NanoBEIR@3 nDCG@10 = 0.5074
tomaarsen/ettin-encoder-150m-multivector-msmarco32k-3e-4-20261014
Feature Extraction • 0.1B • Updated • 94Note NanoBEIR@3 nDCG@10 = 0.5548
tomaarsen/ettin-encoder-150m-reranker-msmarco32k-1e-4-20261014
Text Ranking • 0.1B • Updated • 56Note NanoBEIR@3 nDCG@10 = 0.5861
tomaarsen/ultrabert-150m-dense-msmarco32k-3e-5-20261014
Sentence Similarity • 0.2B • Updated • 56Note NanoBEIR@3 nDCG@10 = 0.5866
tomaarsen/ultrabert-150m-multivector-msmarco32k-1e-4-20261014
Feature Extraction • 0.2B • Updated • 59Note NanoBEIR@3 nDCG@10 = 0.5913
tomaarsen/ultrabert-150m-reranker-msmarco32k-3e-5-20261014
Text Ranking • 0.2B • Updated • 98Note NanoBEIR@3 nDCG@10 = 0.6403
tomaarsen/modernbert-large-dense-msmarco32k-1e-4-20261014
Sentence Similarity • 0.4B • Updated • 45Note NanoBEIR@3 nDCG@10 = 0.5305
tomaarsen/modernbert-large-multivector-msmarco32k-3e-4-20261014
Feature Extraction • 0.4B • Updated • 97Note NanoBEIR@3 nDCG@10 = 0.6022
tomaarsen/modernbert-large-reranker-msmarco32k-1e-4-20261014
Text Ranking • 0.4B • Updated • 47Note NanoBEIR@3 nDCG@10 = 0.6171
tomaarsen/ettin-encoder-400m-dense-msmarco32k-1e-4-20261014
Sentence Similarity • 0.4B • Updated • 44Note NanoBEIR@3 nDCG@10 = 0.5708
tomaarsen/ettin-encoder-400m-multivector-msmarco32k-1e-4-20261014
Feature Extraction • 0.4B • Updated • 102Note NanoBEIR@3 nDCG@10 = 0.6416
tomaarsen/ettin-encoder-400m-reranker-msmarco32k-3e-5-20261014
Text Ranking • 0.4B • Updated • 60Note NanoBEIR@3 nDCG@10 = 0.6422
tomaarsen/ultrabert-380m-dense-msmarco32k-3e-5-20261014
Sentence Similarity • 0.4B • Updated • 48Note NanoBEIR@3 nDCG@10 = 0.6048
tomaarsen/ultrabert-380m-multivector-msmarco32k-3e-5-20261014
Feature Extraction • 0.4B • Updated • 45Note NanoBEIR@3 nDCG@10 = 0.6710
tomaarsen/ultrabert-380m-reranker-msmarco32k-3e-5-20261014
Text Ranking • 0.4B • Updated • 57Note NanoBEIR@3 nDCG@10 = 0.6705
-
tomaarsen/modernbert-base-dense-msmarco32k-3e-4-20261014
Sentence Similarity • 0.1B • Updated • 44 -
tomaarsen/modernbert-base-dense-msmarco32k-3e-5-20261014
Sentence Similarity • 0.1B • Updated • 37 -
tomaarsen/modernbert-base-multivector-msmarco32k-1e-4-20261014
Feature Extraction • 0.1B • Updated • 45 -
tomaarsen/modernbert-base-multivector-msmarco32k-3e-5-20261014
Feature Extraction • 0.1B • Updated • 46 -
tomaarsen/modernbert-base-reranker-msmarco32k-3e-4-20261014
Text Ranking • 0.1B • Updated • 44 -
tomaarsen/modernbert-base-reranker-msmarco32k-3e-5-20261014
Text Ranking • 0.1B • Updated • 49 -
tomaarsen/ettin-encoder-150m-dense-msmarco32k-3e-4-20261014
Sentence Similarity • 0.1B • Updated • 39 -
tomaarsen/ettin-encoder-150m-dense-msmarco32k-3e-5-20261014
Sentence Similarity • 0.1B • Updated • 42 -
tomaarsen/ettin-encoder-150m-multivector-msmarco32k-1e-4-20261014
Feature Extraction • 0.1B • Updated • 90 -
tomaarsen/ettin-encoder-150m-multivector-msmarco32k-3e-5-20261014
Feature Extraction • 0.1B • Updated • 47 -
tomaarsen/ettin-encoder-150m-reranker-msmarco32k-3e-4-20261014
Text Ranking • 0.1B • Updated • 45 -
tomaarsen/ettin-encoder-150m-reranker-msmarco32k-3e-5-20261014
Text Ranking • 0.1B • Updated • 41 -
tomaarsen/ultrabert-150m-dense-msmarco32k-1e-4-20261014
Sentence Similarity • 0.2B • Updated • 42 -
tomaarsen/ultrabert-150m-dense-msmarco32k-3e-4-20261014
Sentence Similarity • 0.2B • Updated • 50 -
tomaarsen/ultrabert-150m-multivector-msmarco32k-3e-4-20261014
Feature Extraction • 0.2B • Updated • 49 -
tomaarsen/ultrabert-150m-multivector-msmarco32k-3e-5-20261014
Feature Extraction • 0.2B • Updated • 45 -
tomaarsen/ultrabert-150m-reranker-msmarco32k-1e-4-20261014
Text Ranking • 0.2B • Updated • 51 -
tomaarsen/ultrabert-150m-reranker-msmarco32k-3e-4-20261014
Text Ranking • 0.2B • Updated • 48 -
tomaarsen/modernbert-large-dense-msmarco32k-3e-4-20261014
Sentence Similarity • 0.4B • Updated • 69 • 1 -
tomaarsen/modernbert-large-dense-msmarco32k-3e-5-20261014
Sentence Similarity • 0.4B • Updated • 43 -
tomaarsen/modernbert-large-multivector-msmarco32k-1e-4-20261014
Feature Extraction • 0.4B • Updated • 67 -
tomaarsen/modernbert-large-multivector-msmarco32k-3e-5-20261014
Feature Extraction • 0.4B • Updated • 47 -
tomaarsen/modernbert-large-reranker-msmarco32k-3e-4-20261014
Text Ranking • 0.4B • Updated • 42 -
tomaarsen/modernbert-large-reranker-msmarco32k-3e-5-20261014
Text Ranking • 0.4B • Updated • 54 -
tomaarsen/ettin-encoder-400m-dense-msmarco32k-3e-4-20261014
Sentence Similarity • 0.4B • Updated • 41 -
tomaarsen/ettin-encoder-400m-dense-msmarco32k-3e-5-20261014
Sentence Similarity • 0.4B • Updated • 45 -
tomaarsen/ettin-encoder-400m-multivector-msmarco32k-3e-4-20261014
Feature Extraction • 0.4B • Updated • 98 -
tomaarsen/ettin-encoder-400m-multivector-msmarco32k-3e-5-20261014
Feature Extraction • 0.4B • Updated • 93 -
tomaarsen/ettin-encoder-400m-reranker-msmarco32k-1e-4-20261014
Text Ranking • 0.4B • Updated • 36 -
tomaarsen/ettin-encoder-400m-reranker-msmarco32k-3e-4-20261014
Text Ranking • 0.4B • Updated • 39 -
tomaarsen/ultrabert-380m-dense-msmarco32k-1e-4-20261014
Sentence Similarity • 0.4B • Updated • 47 -
tomaarsen/ultrabert-380m-dense-msmarco32k-3e-4-20261014
Sentence Similarity • 0.4B • Updated • 44 -
tomaarsen/ultrabert-380m-multivector-msmarco32k-1e-4-20261014
Feature Extraction • 0.4B • Updated • 59 -
tomaarsen/ultrabert-380m-multivector-msmarco32k-3e-4-20261014
Feature Extraction • 0.4B • Updated • 43 -
tomaarsen/ultrabert-380m-reranker-msmarco32k-1e-4-20261014
Text Ranking • 0.4B • Updated • 40 -
tomaarsen/ultrabert-380m-reranker-msmarco32k-3e-4-20261014
Text Ranking • 0.4B • Updated • 43
tomaarsen/ettin-encoder-150m-reranker-msmarco524k-1e-4-20261014
Text Ranking • 0.1B • Updated • 109Note NanoBEIR@3 nDCG@10 = 0.6408 (524k triplets)
tomaarsen/modernbert-base-reranker-msmarco524k-1e-4-20261014
Text Ranking • 0.1B • Updated • 107Note NanoBEIR@3 nDCG@10 = 0.6147 (524k triplets)
tomaarsen/ultrabert-150m-reranker-msmarco524k-3e-5-20261014
Text Ranking • 0.2B • Updated • 138Note NanoBEIR@3 nDCG@10 = 0.6393 (524k triplets)