Upload Deployment

Upload & Deployment

Upload trained models to HuggingFace with optional GGUF conversion and model card generation.

For cloud training, provider-native storage remains the source of truth. Hugging Face Hub publishing is optional and only applies to final_model.

Quick Reference

Save Strategies

GGUF Quantizations

Key Directories

• .skills/upload-deployment/scripts/upload_model.py — Generic upload entry point
• scripts/cloud_gguf_convert.py — Cloud GGUF conversion CLI (download → convert → upload)
• Trainers/recipes/gguf_conversion.yaml — HF Jobs recipe (target: cloud) for cloud GGUF conversion
• shared/upload/ — Upload orchestrator and strategies
• shared/upload/converters/ — GGUF and WebGPU converters
• shared/model_loading/ — Model loading and LoRA merge utilities

Progressive Reference

Load the specific reference you need:

Common Patterns

Standard upload after SFT:

python3 .skills/upload-deployment/scripts/upload_model.py \
  Trainers/sft/sft_output/TIMESTAMP/final_model \
  username/model-name \
  --save-method merged_16bit \
  --create-gguf

Merge LoRA for GRPO continuation:

# Use shared merge utility
./run.sh → Merge LoRA
# Or the GRPO trainer auto-merges when lora_path is set in config

Cloud GGUF conversion (when local RAM is insufficient):

# 1. Upload merged model to HF first (if not already there)
# 2. Edit env vars in the job YAML or override at runtime:
#    GGUF_MODEL_REPO: the HF repo with the merged model
#    GGUF_QUANT_TYPE: q8_0, q5_k_m, or q4_k_m
python tuner.py cloud-run --job-config Trainers/recipes/gguf_conversion.yaml --yes
# 3. GGUF is uploaded back to the same HF repo under gguf/

Cloud GGUF conversion (direct script, outside cloud-run):

python scripts/cloud_gguf_convert.py \
  --model-repo user/model-name \
  --quant q8_0 \
  --upload-to user/model-name

Upload with evaluation results:

# Evaluate first
python -m Evaluator.cli --backend unsloth --model path/to/model \
  --lineage eval_lineage.json --upload-to-hf user/model --update-model-card

Output Structure

After upload, HuggingFace repo contains:

username/model-name/
├── lora/                      # LoRA adapters (if --save-method lora)
├── merged-16bit/              # Full model (if merged_16bit)
├── gguf/                      # GGUF quantizations (if --create-gguf)
│   ├── model-Q4_K_M.gguf
│   ├── model-Q5_K_M.gguf
│   ├── model-Q8_0.gguf
│   └── model-mmproj.gguf     # Vision projector (VL models only)
├── upload_manifest.json       # Upload metadata
├── training_lineage.json      # Training provenance
└── README.md                  # Auto-generated model card

Cloud artifact policy:

• Default: artifacts stay in provider-native storage
• hf_jobs: Hugging Face Bucket
• modal: Modal Volume
• runpod: RunPod Network Volume
• Optional publish: only final_model is pushed to the target HF repo when enabled

Environment Variables

HF_TOKEN=hf_...                       # Required for uploads

Tips

• Always use merged_16bit as the source for GGUF conversion (best quality)
• The reliable GGUF converter merges LoRA once, then creates all quants (~10 min saved)
• Vision-language models auto-get an mmproj.gguf for the vision projector
• On macOS, bucket-backed cloud adapters are often easiest to handle one model at a time: pull the final_model, merge locally, create the quant you actually need first, then clean temp files before moving to the next model
• If the local machine lacks unsloth, a plain transformers + peft merge venv is an acceptable fallback for text models before llama.cpp conversion
• For merged local models, call the lower-level llama.cpp conversion path directly; the current reliable converter's top-level convert() flow assumes it starts from a LoRA adapter
• LM Studio on this repo owner's Mac uses ~/.lmstudio/models/<publisher>/<model-folder>/; placing the .gguf there plus an optional config.json is enough for local testing after refresh/restart
• Qwen 3.5 adapters may need a ConditionalGeneration merge path instead of AutoModelForCausalLM; if the adapter keys live under language_model.*, inspect the base architecture before merging
• If llama.cpp says a merged model architecture is unsupported, update the local Trainers/llama.cpp checkout before retrying conversion; newer model families are often converter-gated rather than merge-gated
• On WSL, temp files use native filesystem to avoid NTFS performance issues
• training_lineage.json is auto-generated — includes model, LoRA, dataset, hardware info
• Use upload_manifest.json to verify what was uploaded
• The upload orchestrator handles everything — prefer ./run.sh → Upload over manual commands
• Cloud jobs never rely on the remote container filesystem as the only copy; inspect provider-native storage first, then publish final_model if needed
• If local GGUF conversion OOMs (common on machines with <32GB RAM), use the cloud GGUF job (cpu-upgrade flavor, 32GB RAM, no GPU needed)
• The cloud GGUF script uses pure Python conversion (llama.cpp convert_hf_to_gguf.py) — no compilation required
• Some models (e.g., Gemma 4) may need tokenizer config patching before conversion — the cloud script handles known quirks automatically