# Minicpm5 Deploy Mlx

> Run MiniCPM5-1B or MiniCPM5-2B natively on Apple Silicon with Apple's MLX framework. Use when the user has an Apple Silicon Mac and asks for "MLX", "mlx_lm", "mlx_lm.convert", "mlx_lm.generate", or wants the fastest path on Apple Silicon.

- Skill: `openbmb/minicpm5-deploy-mlx` (Agent Skill)
- Install (CLI): `npx skillmds@latest add openbmb/minicpm5-deploy-mlx`
- Raw SKILL.md: https://api.skillmd.com/api/skills/openbmb/minicpm5-deploy-mlx/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: DevOps & Infra
- Author: OpenBMB (https://skillmd.com/u/openbmb)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/openbmb/minicpm5-deploy-mlx

---


# Deploy MiniCPM5-1B and MiniCPM5-2B with MLX (Apple Silicon)

Apple's on-device tensor framework. Highest throughput on M-series. Stays inside one Python process — no separate server, no `llama.cpp` build chain.

## Required input

| Var | Example | Default |
| --- | --- | --- |
| `MLX_REPO` | `openbmb/MiniCPM5-2B-MLX` (pre-converted 4-bit affine) | required; `openbmb/MiniCPM5-1B-MLX` also works |
| OR `HF_REPO` + `QUANT` | `openbmb/MiniCPM5-2B`, `4bit` or `bf16` | for local conversion |
| `MAX_TOKENS` | `200` | `200` |

## Steps

### 1. Install (once)

```bash
pip install "mlx-lm>=0.31" "gguf"
```

### 2A. Use a pre-converted MLX repo (recommended when available)

```bash
mlx_lm.generate --model "${MLX_REPO}" \
    --prompt "<|im_start|>user
1+1=?<|im_end|>
<|im_start|>assistant
" \
    --max-tokens ${MAX_TOKENS} --temp 1.0 --top-p 0.95
```

### 2B. Convert from a HF checkpoint locally (advanced)

Use `mlx_lm.convert` only if you have a self-trained HF fp16 checkpoint:

```bash
HF=/path/to/your-fp16-hf

# Convert: bf16
mlx_lm.convert --hf-path "$HF" --mlx-path ./minicpm5-mlx-bf16

# Convert: 4-bit (smaller / faster)
mlx_lm.convert --hf-path "$HF" --mlx-path ./minicpm5-mlx-q4 -q --q-bits 4
```

Then run as in 2A.

### 3. Validate

The reply should contain `"2"` for `1+1=?`.

## OpenAI-compatible server (mlx-lm)

```bash
mlx_lm.server --model "${MLX_REPO}" --host 127.0.0.1 --port 8000

curl http://127.0.0.1:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
        "model": "default",
        "messages": [{"role":"user","content":"1+1=?"}],
        "temperature": 1.0, "top_p": 0.95, "max_tokens": 64
    }'
```

## Common pitfalls

- **Slow first generate**: MLX JIT-compiles kernels on first call (~5-10 s); subsequent calls hit the warm cache.
- **Model runs past `<|im_end|>`**: only happens on `mlx-lm < 0.31` (older versions ignore multi-id `eos_token_id` lists). Upgrade, or pass `--extra-eos-token "<|im_end|>"` as a manual override — `<|im_end|>` is token id 130073 and is already listed in `generation_config.json` on 0.31+.

## When NOT to use

- Not on Apple Silicon → `minicpm5-deploy-llama-cpp` (CPU/CUDA) or `minicpm5-deploy-vllm` (CUDA)
- Want a desktop GUI → `minicpm5-deploy-lmstudio` (LM Studio bundles an MLX runtime)
- Want one-line CLI run → `minicpm5-deploy-ollama`

## Reference

[`docs/deployment/mlx.md`](../../docs/deployment/mlx.md)