Huggingface

Hugging Face Skill

Purpose

Provides structured workflows for interacting with the Hugging Face ecosystem: Hub repository search, model loading and inference, dataset management, PEFT/LoRA fine-tuning, and Spaces deployment. Integrates via MCP tools when available and falls back to Python APIs and the huggingface_hub CLI.

When to Invoke

Skill({ skill: 'huggingface' });

Invoke when:

• Searching for models, datasets, or Spaces on the Hub
• Loading and running inference with Transformers models
• Managing datasets with the datasets library
• Fine-tuning models with PEFT/LoRA
• Deploying or querying Hugging Face Spaces
• Selecting the right model for a task (NLP, vision, audio, multimodal)

MCP Tool Integration

When the Hugging Face MCP server is available, prefer MCP tools over direct API calls.

Hub Search

Search for models by task or keyword:

// Search models
mcp__claude_ai_Hugging_Face__hub_repo_search({
  query: 'text-classification',
  repo_type: 'model',
  limit: 10,
});

// Get detailed repo info
mcp__claude_ai_Hugging_Face__hub_repo_details({
  repo_id: 'bert-base-uncased',
  repo_type: 'model',
});

Expected output: JSON list of repos with modelId, downloads, likes, tags, pipeline_tag, and lastModified.

Verify: Confirm pipeline_tag matches the intended task before proceeding.

Paper and Space Search

// Search arXiv papers on the Hub
mcp__claude_ai_Hugging_Face__paper_search({
  query: 'instruction tuning language models',
  limit: 5,
});

// Search Spaces
mcp__claude_ai_Hugging_Face__space_search({
  query: 'stable diffusion',
  limit: 5,
});

Documentation Search

// Search HF docs
mcp__claude_ai_Hugging_Face__hf_doc_search({
  query: 'AutoModelForCausalLM from_pretrained',
  library: 'transformers',
});

Model Loading & Inference

Standard Transformers Pipeline

from transformers import pipeline

# Text generation
generator = pipeline(
    "text-generation",
    model="meta-llama/Llama-3.2-1B-Instruct",
    device_map="auto",
    torch_dtype="auto",
)
result = generator("What is the capital of France?", max_new_tokens=100)
print(result[0]["generated_text"])

Verify: result[0]["generated_text"] contains coherent continuation. Check device_map resolves to GPU if available.

AutoModel Pattern (Explicit Control)

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "microsoft/Phi-3-mini-4k-instruct"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    attn_implementation="flash_attention_2",  # if available
)

inputs = tokenizer("Hello, world!", return_tensors="pt").to(model.device)
with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=50)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Inference API (Serverless)

from huggingface_hub import InferenceClient

client = InferenceClient(token="hf_...")  # or use HF_TOKEN env var

# Text generation
result = client.text_generation(
    "Tell me a joke",
    model="mistralai/Mistral-7B-Instruct-v0.3",
    max_new_tokens=200,
)
print(result)

# Chat completion (OpenAI-compatible)
response = client.chat_completion(
    model="meta-llama/Llama-3.1-8B-Instruct",
    messages=[{"role": "user", "content": "Explain quantum entanglement"}],
    max_tokens=300,
)
print(response.choices[0].message.content)

Verify: Check response.choices[0].finish_reason == "stop" for complete generation.

Dataset Management

Loading Datasets

from datasets import load_dataset

# Public dataset
ds = load_dataset("squad", split="train")
print(ds.column_names)  # ['id', 'title', 'context', 'question', 'answers']

# With streaming (large datasets)
ds_stream = load_dataset("allenai/c4", "en", split="train", streaming=True)
sample = next(iter(ds_stream))

# From Hub with specific config
ds = load_dataset("glue", "mrpc", split={"train": "train", "val": "validation"})

Verify: len(ds) returns expected row count. ds.features shows correct schema.

Creating and Pushing Datasets

from datasets import Dataset, DatasetDict
from huggingface_hub import HfApi

# Create from dict
data = {"text": ["Hello", "World"], "label": [0, 1]}
ds = Dataset.from_dict(data)

# Push to Hub
ds.push_to_hub("username/my-dataset", private=True, token="hf_...")

# Verify upload
api = HfApi()
info = api.dataset_info("username/my-dataset")
print(f"Rows: {info.cardData.get('dataset_info', {}).get('splits', {})}")

Dataset Preprocessing

from datasets import load_dataset
from transformers import AutoTokenizer

ds = load_dataset("imdb", split="train")
tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased")

def tokenize(batch):
    return tokenizer(batch["text"], truncation=True, padding="max_length", max_length=512)

ds_tokenized = ds.map(tokenize, batched=True, remove_columns=["text"])
ds_tokenized.set_format(type="torch", columns=["input_ids", "attention_mask", "label"])

PEFT / LoRA Fine-Tuning

LoRA Configuration

from peft import LoraConfig, get_peft_model, TaskType

lora_config = LoraConfig(
    r=16,                          # rank — higher = more parameters
    lora_alpha=32,                 # scaling factor
    target_modules=["q_proj", "v_proj"],  # layers to apply LoRA to
    lora_dropout=0.05,
    bias="none",
    task_type=TaskType.CAUSAL_LM,
)

model = get_peft_model(model, lora_config)
model.print_trainable_parameters()
# Expected: trainable params: ~X MB / total: ~Y GB (<1% of total)

QLoRA (4-bit Quantized LoRA)

from transformers import BitsAndBytesConfig, AutoModelForCausalLM
from peft import prepare_model_for_kbit_training

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype="bfloat16",
    bnb_4bit_use_double_quant=True,
)

model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.1-8B",
    quantization_config=bnb_config,
    device_map="auto",
)
model = prepare_model_for_kbit_training(model)

SFT with TRL (2025 Best Practices)

Use conversational message format for modern SFT workflows:

from trl import SFTTrainer, SFTConfig
from datasets import load_dataset

# 2025: Use conversational format with system prompt
dataset = load_dataset("HuggingFaceH4/ultrachat_200k", split="train_sft")

training_args = SFTConfig(
    output_dir="./sft-output",
    num_train_epochs=3,
    per_device_train_batch_size=4,
    gradient_accumulation_steps=4,
    learning_rate=2e-4,
    bf16=True,                         # prefer bf16 over fp16 for stability
    logging_steps=10,
    save_strategy="epoch",
    # Performance: pack sequences to fill context window (reduces padding waste)
    packing=True,
    max_seq_length=2048,
    # Train only on assistant responses, not on user/system tokens
    dataset_kwargs={"skip_prepare_dataset": False},
    push_to_hub=True,
    hub_model_id="username/my-finetuned-model",
)

trainer = SFTTrainer(
    model=model,
    args=training_args,
    train_dataset=dataset,
    peft_config=lora_config,
)
trainer.train()
trainer.push_to_hub()

Performance Optimization (Liger Kernels + Flash Attention):

# Significant speedup for supported architectures (Llama, Mistral, Phi)
from liger_kernel.transformers import apply_liger_kernel_to_llama

# Apply before model loading
apply_liger_kernel_to_llama()

model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.1-8B",
    torch_dtype=torch.bfloat16,
    device_map="auto",
    attn_implementation="flash_attention_2",  # 2-4x throughput improvement
)

Verify: Loss decreases from epoch 1 to 3. trainer.state.log_history[-1]["train_loss"] should be < initial loss. Training time with packing + Flash Attention + Liger Kernels is typically 5-20x faster than naive baseline.

Merging Adapters

from peft import PeftModel

# Load base + adapter
base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-8B")
model = PeftModel.from_pretrained(base_model, "username/my-lora-adapter")

# Merge and unload (creates standalone model)
merged = model.merge_and_unload()
merged.save_pretrained("./merged-model")

Spaces Deployment

Gradio Space

# app.py for a Gradio Space
import gradio as gr
from transformers import pipeline

pipe = pipeline("text-classification", model="distilbert-base-uncased-finetuned-sst-2-english")

def classify(text):
    result = pipe(text)[0]
    return f"{result['label']}: {result['score']:.3f}"

demo = gr.Interface(fn=classify, inputs="text", outputs="text", title="Sentiment Classifier")
demo.launch()

requirements.txt:

transformers>=4.40.0
torch>=2.2.0
gradio>=4.0.0

Querying Existing Spaces

from gradio_client import Client

client = Client("hf-audio/whisper-large-v3")
result = client.predict(
    audio="path/to/audio.wav",
    api_name="/predict",
)
print(result)

Deploying via API

from huggingface_hub import HfApi

api = HfApi(token="hf_...")

# Create Space
api.create_repo(
    repo_id="username/my-space",
    repo_type="space",
    space_sdk="gradio",
    private=False,
)

# Upload files
api.upload_folder(
    folder_path="./my-space-app",
    repo_id="username/my-space",
    repo_type="space",
)

Verify: Space appears at https://huggingface.co/spaces/username/my-space and status is RUNNING.

Model Selection Guide

By Task

Selection Criteria

1. Check downloads and likes — higher = more community validation
2. Check pipeline_tag — must match intended task
3. Check model card — look for benchmark scores and limitations
4. Check license — apache-2.0 or mit for commercial use; llama license has restrictions
5. Check size — 7B fits on 16GB VRAM with fp16; use 4-bit for smaller GPUs

# CLI: search by task
huggingface-cli repo search --filter pipeline_tag:text-generation --limit 10

Evaluation Strategy

Before fine-tuning, evaluate whether prompting alone solves your problem. Fine-tuning is justified for: domain-specific knowledge injection, controlled output style, hallucination reduction in narrow domains, and specialized task optimization at scale.

Evaluate fine-tuned models in production-like conditions:

# Serve the fine-tuned model with TGI or vLLM for realistic latency testing
docker run --gpus all -p 8080:80 \
  ghcr.io/huggingface/text-generation-inference:latest \
  --model-id username/my-finetuned-model

# Run evaluation harness against the served model
lm_eval --model openai-chat-completions \
  --model_args base_url=http://localhost:8080/v1,model=username/my-finetuned-model \
  --tasks mmlu,hellaswag \
  --output_path ./eval-results/

Verify: Perplexity on held-out validation set decreases. Task-specific benchmark scores match or exceed baseline model.

Anti-Patterns

• Never hardcode HF tokens — use HF_TOKEN env var or huggingface-cli login
• Never load full model for inference-only on CPU — use device_map="auto" or Inference API
• Never skip attn_implementation — for supported models, flash_attention_2 gives 2-4x speedup
• Never ignore tokenizer warnings — padding/truncation mismatches cause silent accuracy drops
• Never push private data to public repos — set private=True or use push_to_hub(private=True)
• Never use pipeline() in production fine-tuning loops — use AutoModel + Trainer for control
• Never merge adapters before evaluation — evaluate PEFT model first, merge only if satisfactory
• Never use model.generate() without max_new_tokens — unbounded generation hangs
• Never skip packing for SFT — packing=True in SFTConfig dramatically reduces training time by filling context windows
• Never use fp16=True when bf16 is available — bfloat16 is more numerically stable for LLM fine-tuning on Ampere+ GPUs
• Never evaluate only on training distribution — use held-out eval set and standard benchmarks via lm-evaluation-harness

Environment Setup

# Install core stack
pip install transformers datasets peft trl accelerate bitsandbytes

# Optional: flash attention (Linux + CUDA only)
pip install flash-attn --no-build-isolation

# Login to Hub
huggingface-cli login  # paste HF_TOKEN when prompted

# Verify GPU
python -c "import torch; print(torch.cuda.is_available(), torch.cuda.get_device_name(0))"

Related Skills

• python-backend-expert — Python project setup and best practices
• debugging — Systematic debugging for training instabilities
• mcp-catalog — MCP server selection and configuration

Search Protocol

Before starting any Hugging Face task, search for existing model loading code and dataset pipelines:

pnpm search:code "from transformers OR from datasets OR InferenceClient OR SFTTrainer"
pnpm search:code "huggingface fine-tuning"

Use Skill({ skill: 'ripgrep' }) to find existing .py training scripts. Use Skill({ skill: 'code-semantic-search' }) to find similar ML pipeline patterns by intent.

Memory Protocol (MANDATORY)

Before starting any task, you must query semantic memory and read recent static memory:

node .claude/lib/memory/memory-search.cjs "huggingface transformers fine-tuning model selection"

Read .claude/context/memory/learnings.md Read .claude/context/memory/decisions.md

Check for prior model selections, known HF API rate limits, tokenizer issues, and CUDA compatibility gotchas.

After completing work, record findings:

• Model selection decisions (why model X over Y) -> Update .claude/context/memory/decisions.md
• HF API quirks, rate limits, token issues -> Append to .claude/context/memory/issues.md
• Training optimization discoveries -> Append to .claude/context/memory/learnings.md

During long tasks: Use .claude/context/memory/active_context.md as scratchpad.

ASSUME INTERRUPTION: Your context may reset. If it's not in memory, it didn't happen.