Instructions to use 0x7o/poemma-12B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use 0x7o/poemma-12B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="0x7o/poemma-12B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("0x7o/poemma-12B")
model = AutoModelForCausalLM.from_pretrained("0x7o/poemma-12B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use 0x7o/poemma-12B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "0x7o/poemma-12B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "0x7o/poemma-12B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/0x7o/poemma-12B

SGLang

How to use 0x7o/poemma-12B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "0x7o/poemma-12B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "0x7o/poemma-12B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "0x7o/poemma-12B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "0x7o/poemma-12B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Unsloth Desktop
Docker Model Runner
How to use 0x7o/poemma-12B with Docker Model Runner:
```
docker model run hf.co/0x7o/poemma-12B
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Жить бы и радоваться,

Не по себе, —

Творческий кризис,

Что это такое?

Я не поэт,

Я интеллектуальный автомат.

Poemma 12B

Language model, fine tuning over Mistral Nemo using ORPO to generate poems in Russian with imitation of the style of some classics.

Usage

A model can write poetry from scratch, or imitate the style of the following authors:

'Марина Цветаева', 'Александр Пушкин', 'Александр Блок',
'Анна Ахматова', 'Владимир Высоцкий', 'Иосиф Бродский',
'Константин Бальмонт', 'Владимир Маяковский', 'Николай Гумилев',
'Федор Тютчев', 'Сергей Есенин', 'Игорь Северянин', 'Осип Мандельштам',
'Агния Барто', 'Евгений Евтушенко', 'Роберт Рождественский',
'Иван Бунин', 'Андрей Дементьев', 'Самуил Маршак'

from transformers import pipeline

messages = [
    {"role": "user", "content": "Напиши стих о размышлениях нейронной сети в стиле Пушкина. Используй высокую лексику."},
]
pipe = pipeline("text-generation", model="0x7o/poemma-12B")
pipe(messages)

Dataset

https://huggingface.co/datasets/0x7o/poemma-10k

Author

https://t.me/kononyuk

Downloads last month: 40

Safetensors

Model size

12B params

Tensor type

BF16

Model tree for 0x7o/poemma-12B

Quantizations

6 models

0x7o
/

poemma-12B

Poemma 12B

Usage

Dataset

Author

Model tree for 0x7o/poemma-12B

Dataset used to train 0x7o/poemma-12B