Genos-m

Genos-m is a foundation model for human-associated microbial genomes. It is trained to model microbial DNA sequences at single-nucleotide resolution and supports ultra-long genomic contexts up to one million tokens.

For instructions, details, benchmarks, and examples, please refer to Genos-m GitHub and paper.

Model Specification

Specification	Genos-m-4.7B
Total parameters	4.7B
Activated parameters	0.33B
Architecture type	MoE
Number of experts	32
Selected experts per token	2
Number of layers	12
Attention hidden size	1024
Number of attention heads	16
Query groups	8
MoE hidden size per expert	4096
Vocabulary size	128 padded
Context length	up to 1M
Training objective	next-token prediction

Training Data

Genos-m was pretrained on curated microbial genome resources, including GTDB R220 representative prokaryotic genomes, public human-associated microbial genomes, in-house high-quality human gut MAGs, and UHGV human gut phage genomes. The final pre-training corpus contains approximately 1.2T tokens and covers 186 phyla, 3,448 families, and 69,056 species. Within this corpus, the retained human-associated prokaryotic subset covers 45 phyla, 585 families, and 12,273 species across major human microbial habitats, including the gut, oral cavity, skin, respiratory tract, and female reproductive tract.

Checkpoints

• HF-Transformers checkpoint: BGI-HangzhouAI/Genos-m-4.7B
• Megatron-LM checkpoint: BGI-HangzhouAI/Genos-m-Megatron-4.7B

License

Genos-m model and code are released under the Apache License 2.0.

Contact

For questions and suggestions, please open an issue.