Instructions to use pottokao/MiniMax-H3-TextEncoder-16GB-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use pottokao/MiniMax-H3-TextEncoder-16GB-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf pottokao/MiniMax-H3-TextEncoder-16GB-GGUF
# Run inference directly in the terminal:
llama cli -hf pottokao/MiniMax-H3-TextEncoder-16GB-GGUF

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf pottokao/MiniMax-H3-TextEncoder-16GB-GGUF
# Run inference directly in the terminal:
llama cli -hf pottokao/MiniMax-H3-TextEncoder-16GB-GGUF

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf pottokao/MiniMax-H3-TextEncoder-16GB-GGUF
# Run inference directly in the terminal:
./build/bin/llama cli -hf pottokao/MiniMax-H3-TextEncoder-16GB-GGUF

Use Docker

docker model run hf.co/pottokao/MiniMax-H3-TextEncoder-16GB-GGUF

Llama new
LM Studio
Jan
Ollama
How to use pottokao/MiniMax-H3-TextEncoder-16GB-GGUF with Ollama:
```
ollama run hf.co/pottokao/MiniMax-H3-TextEncoder-16GB-GGUF
```
Unsloth Desktop

How to use pottokao/MiniMax-H3-TextEncoder-16GB-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf pottokao/MiniMax-H3-TextEncoder-16GB-GGUF

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "pottokao/MiniMax-H3-TextEncoder-16GB-GGUF"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use pottokao/MiniMax-H3-TextEncoder-16GB-GGUF with Docker Model Runner:
```
docker model run hf.co/pottokao/MiniMax-H3-TextEncoder-16GB-GGUF
```

Lemonade

How to use pottokao/MiniMax-H3-TextEncoder-16GB-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull pottokao/MiniMax-H3-TextEncoder-16GB-GGUF

Run and chat with the model

lemonade run user.MiniMax-H3-TextEncoder-16GB-GGUF-{{QUANT_TAG}}

List all available models

lemonade list

Hermes Agent

How to use pottokao/MiniMax-H3-TextEncoder-16GB-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf pottokao/MiniMax-H3-TextEncoder-16GB-GGUF

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default pottokao/MiniMax-H3-TextEncoder-16GB-GGUF

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use pottokao/MiniMax-H3-TextEncoder-16GB-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf pottokao/MiniMax-H3-TextEncoder-16GB-GGUF

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "pottokao/MiniMax-H3-TextEncoder-16GB-GGUF" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

pottokao commited on 23 days ago

Commit

35364a4

verified ·

1 Parent(s): 82ef732

16 GB cards: use 11.5G; 12.5G/13.6G OOM by default, --reserve-vram 1.5 workaround (encoder step only tested)

Browse files

Files changed (1) hide show

README.md +2 -1

README.md CHANGED Viewed

@@ -43,7 +43,8 @@ included yet.
 - The visual tower (F16) is merged in because ComfyUI recognizes the H3 encoder by `visual.deepstack_merger_list.*` +
   `model.layers.49.*`. Its 5-D `patch_embed` weight is stored the ComfyUI-GGUF way (flattened +
   `comfy.gguf.orig_shape`), so current `gguf` versions read it.
-- **Tested on a 16 GB card** (RTX 5070 Ti, ComfyUI, H3 FL2VA turbo 4-step DiT, 832×480 × 61 frames, `--disable-dynamic-vram --reserve-vram 0.15`): the **11.5G** file runs (whole clip ~45–65 s, GPU peak ≈ 15.0 GB). The **12.5G and 13.6G files ran out of memory** while loading the encoder on that card — use them on 20 GB+ cards. ComfyUI's default dynamic VRAM may let them offload on 16 GB; not tested.
 ## How the sizes were spent

 - The visual tower (F16) is merged in because ComfyUI recognizes the H3 encoder by `visual.deepstack_merger_list.*` +
   `model.layers.49.*`. Its 5-D `patch_embed` weight is stored the ComfyUI-GGUF way (flattened +
   `comfy.gguf.orig_shape`), so current `gguf` versions read it.
+- **16 GB cards → use the 11.5G file.** Tested on an RTX 5070 Ti 16 GB (ComfyUI, H3 FL2VA turbo 4-step DiT, euler + sigma shift 12/3, 1344×768 × 61 frames): full clips render with default settings, ~75 s per clip, GPU peak ≈ 15 GB.
+- **12.5G / 13.6G on a 16 GB card:** with ComfyUI's default settings the encoder runs out of memory (`torch.OutOfMemoryError`, no video). Starting ComfyUI with `--reserve-vram 1.5` makes it keep part of the encoder in system RAM: the encoding step then passes (first prompt 12 s / 19 s, later ones ~4 s), but a full clip with that setting was **not** tested — the DiT may then be pushed to offload too. On 20 GB+ cards they load normally.
 ## How the sizes were spent