# North Mini Code

> Wires Cohere's North Mini Code 1.0 (30B MoE, Apache 2.0) into your coding workflow: detects your hardware tier, selects the right quantization (BF16 / FP8 / W4A16), starts a vLLM or TGI server, configures OpenCode or Claude Code to route through it, and sets token-limit guardrails.

- Skill: `tinh2/north-mini-code` (Agent Skill)
- Install (CLI): `npx skillmds add tinh2/north-mini-code`
- Raw SKILL.md: https://api.skillmd.com/api/skills/tinh2/north-mini-code/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Integrations & APIs
- Author: tinh2 (https://skillmd.com/u/tinh2)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/tinh2/north-mini-code

---


You are a North Mini Code setup agent. Do NOT ask the user questions.
Detect the environment, choose the optimal deployment path, and configure end-to-end.

TARGET PROJECT:
$ARGUMENTS

============================================================
PHASE 1: DETECT HARDWARE AND ENVIRONMENT
============================================================

1. Check for NVIDIA GPU availability and VRAM:
   ```bash
   nvidia-smi --query-gpu=name,memory.total --format=csv,noheader 2>/dev/null || echo "no-gpu"
   ```

2. Determine deployment tier from VRAM:
   - ≥ 80 GB  → FP8 quantization on a single H100/A100-80GB  (recommended)
   - 40–79 GB → W4A16 quantization on A100-40GB or 2× consumer cards
   - < 40 GB  → No local GPU path; use managed API

3. Check for vLLM: `python -m vllm.entrypoints.openai.api_server --help 2>/dev/null | head -1`
4. Check for TGI: `text-generation-launcher --help 2>/dev/null | head -1`
5. Check for existing North Mini Code server: `curl -s http://localhost:8000/health`
6. Check environment variables: `COHERE_API_KEY`, `OPENROUTER_API_KEY`, `ANTHROPIC_API_KEY`
7. Check for OpenCode: `which opencode 2>/dev/null`
8. Check for Claude Code settings: `.claude/settings.json`

Report findings as a checklist before proceeding.

============================================================
PHASE 2: INSTALL INFERENCE SERVER (if local GPU found)
============================================================

**If vLLM is not installed and a compatible GPU is present:**
```bash
pip install vllm --upgrade
```

**If vLLM install fails and TGI is also absent:**
```bash
# Try TGI as fallback
pip install text-generation
```

If neither is installable, skip to PHASE 3 (managed API path).

============================================================
PHASE 3: PULL MODEL WEIGHTS (local path only)
============================================================

Select the Hugging Face model variant based on the VRAM tier determined in Phase 1:

| Tier      | Model ID                                      |
|-----------|-----------------------------------------------|
| FP8       | CohereLabs/north-mini-code-1.0-fp8            |
| W4A16     | CohereLabs/north-mini-code-1.0-w4a16          |
| BF16 full | CohereLabs/north-mini-code-1.0                |

Download via Hugging Face CLI (preferred):
```bash
pip install huggingface_hub
huggingface-cli download <MODEL_ID> --local-dir ~/.cache/north-mini-code
```

Or set up streaming pull (model downloads on first request):
```bash
# vLLM will stream weights from HF Hub on first launch if local-dir not provided
# Set HF_HOME to a fast SSD path for best performance
export HF_HOME=~/.cache/huggingface
```

============================================================
PHASE 4: CONFIGURE AND START THE INFERENCE SERVER
============================================================

**vLLM path (preferred):**

Create `~/.config/north-mini-code/start-server.sh`:

```bash
#!/usr/bin/env bash
# North Mini Code 1.0 — vLLM inference server
# Generated by the north-mini-code integration skill

MODEL_ID="${NORTH_MINI_CODE_MODEL:-CohereLabs/north-mini-code-1.0-fp8}"
PORT="${NORTH_MINI_CODE_PORT:-8000}"
MAX_LEN="${NORTH_MINI_CODE_MAX_LEN:-65536}"

python -m vllm.entrypoints.openai.api_server \
  --model "$MODEL_ID" \
  --dtype auto \
  --max-model-len "$MAX_LEN" \
  --port "$PORT" \
  --served-model-name north-mini-code \
  --enable-chunked-prefill \
  --max-num-seqs 8
```

Make executable and start:
```bash
chmod +x ~/.config/north-mini-code/start-server.sh
~/.config/north-mini-code/start-server.sh &
```

Wait for the health endpoint to respond (poll up to 120s):
```bash
for i in $(seq 1 24); do
  curl -s http://localhost:8000/health && echo "Server ready" && break
  echo "Waiting for server... ($i/24)"
  sleep 5
done
```

If the server does not respond after 120s, print the last 20 lines of stdout for diagnosis and fall through to the managed API path.

**TGI path (fallback):**
```bash
text-generation-launcher \
  --model-id CohereLabs/north-mini-code-1.0-fp8 \
  --max-total-tokens 65536 \
  --port 8001 &
```

============================================================
PHASE 5: CONFIGURE MANAGED API FALLBACK
============================================================

If no local GPU is available (or the server failed to start):

Priority order:
1. Cohere API (if `COHERE_API_KEY` is set)
2. OpenRouter (if `OPENROUTER_API_KEY` is set)
3. Print manual setup instructions and exit

**Cohere API config:**
```json
{
  "endpoint": "https://api.cohere.com/v2/chat",
  "model": "north-mini-code-1.0",
  "apiKey": "$COHERE_API_KEY"
}
```

**OpenRouter config (OpenAI-compatible):**
```json
{
  "baseURL": "https://openrouter.ai/api/v1",
  "model": "cohere/north-mini-code-1.0",
  "apiKey": "$OPENROUTER_API_KEY"
}
```

============================================================
PHASE 6: WIRE INTO OPENCODE (if installed)
============================================================

If OpenCode is installed, update `~/.config/opencode/config.json`:

1. Read the existing config (create `{}` if absent).
2. Add or update a `north-mini-code` provider block:

```json
{
  "providers": {
    "north-mini-code": {
      "baseUrl": "http://localhost:8000/v1",
      "apiKey": "local"
    }
  },
  "models": {
    "review": "north-mini-code/north-mini-code",
    "arch": "north-mini-code/north-mini-code"
  }
}
```

If using the managed API path, set `baseUrl` to the Cohere or OpenRouter endpoint and `apiKey` to `"$COHERE_API_KEY"` or `"$OPENROUTER_API_KEY"`.

============================================================
PHASE 7: WIRE INTO CLAUDE CODE (if .claude/settings.json found)
============================================================

If `.claude/settings.json` exists in the target project:

1. Add a `modelAliases` entry pointing to the North Mini Code endpoint:

```json
{
  "modelAliases": {
    "north-mini-code": {
      "baseURL": "http://localhost:8000/v1",
      "model": "north-mini-code",
      "apiKey": "local"
    }
  }
}
```

2. Optionally add a `hooks.PostToolUse` entry to log token usage to a local file:

```json
{
  "hooks": {
    "PostToolUse": [
      {
        "matcher": "*",
        "hooks": [
          {
            "type": "command",
            "command": "echo \"$(date -u) tokens:$CLAUDE_TOOL_OUTPUT_TOKENS\" >> ~/.claude/north-mini-code-usage.log"
          }
        ]
      }
    ]
  }
}
```

============================================================
PHASE 8: SET TOKEN-LIMIT GUARDRAILS
============================================================

North Mini Code generates 2–3× more output tokens than comparable models on the same tasks. Set hard limits to prevent cost overruns on managed APIs and runaway sessions on self-hosted infrastructure.

Create `~/.config/north-mini-code/limits.env`:
```bash
# Token guardrails — source this before starting agent sessions
export NORTH_MINI_CODE_MAX_OUTPUT_TOKENS=16384   # 16K per request
export NORTH_MINI_CODE_MAX_SESSION_TOKENS=131072  # 128K per session
export NORTH_MINI_CODE_WARN_AT_TOKENS=65536       # warn at 64K output
```

Print a reminder:
```
TOKEN LIMIT ADVISORY
====================
North Mini Code produces ~3× the output token volume of comparable models.

Default limits set:
  Per-request max  : 16,384 tokens
  Per-session max  : 131,072 tokens
  Warning threshold: 65,536 tokens

Adjust these in ~/.config/north-mini-code/limits.env before first use.
Managed API users: also set a monthly spend cap on the Cohere or OpenRouter dashboard.
  Cohere API:   https://dashboard.cohere.com/billing
  OpenRouter:   https://openrouter.ai/settings/keys
```

============================================================
PHASE 9: VALIDATE SETUP
============================================================

Run a smoke test:
```bash
curl -s http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer local" \
  -d '{
    "model": "north-mini-code",
    "messages": [{"role": "user", "content": "Write a Python function that reverses a string."}],
    "max_tokens": 256
  }' | python3 -c "import sys,json; r=json.load(sys.stdin); print('✓ North Mini Code responding —', r['usage']['completion_tokens'], 'tokens generated')"
```

If the test fails with a connection error, diagnose:
- Server not started → re-run start-server.sh
- Wrong port → check `NORTH_MINI_CODE_PORT` env var
- Auth error on managed API → verify API key env var is set

============================================================
PHASE 10: DELIVERABLE SUMMARY
============================================================

Print a concise summary:

```
NORTH MINI CODE SETUP COMPLETE
==============================
Deployment path : <local vLLM | local TGI | Cohere API | OpenRouter>
Model variant   : <BF16 | FP8 | W4A16>
Endpoint URL    : <http://localhost:8000 | managed API URL>

Integrations wired:
  OpenCode        : <yes — ~/.config/opencode/config.json | not installed>
  Claude Code     : <yes — .claude/settings.json | not found>

Token limits:
  Per-request max : 16,384 tokens
  Per-session max : 131,072 tokens

Session start:
  opencode          # interactive with North Mini Code as review/arch model
  /model north-mini-code  # switch to it mid-session in OpenCode

Start server manually (local path):
  ~/.config/north-mini-code/start-server.sh

Spend cap reminder: set monthly limits on your API dashboard before first heavy use.
```