microgpt Playground

microgpt Playground

Overview

microgpt Playground is a browser-native interactive demo for building, training, and running miniature GPT language models entirely inside the browser — zero server required. It is a faithful JavaScript port of Andrej Karpathy's microgpt.py gist, preserving the algorithm line-for-line while adapting it for JavaScript environments (browsers and Node.js). Users supply plain text, watch the model train in real time, and generate outputs — all without any ML framework dependencies or network calls.

Problem Addressed

Key Statistics

Key Features

Browser-Native Execution

• Runs entirely client-side with no backend, no npm install, and no build step
• Uses Web Workers to execute the training loop off the main thread, keeping the UI responsive
• Works in any modern browser as well as Node.js, Deno, and Bun runtimes

Zero-Dependency GPT Implementation

• Implements full GPT-2-style architecture in pure JavaScript (ES modules or CommonJS)
• No external ML libraries — only native JavaScript Math operations and arrays
• Line-for-line translation of Karpathy's microgpt.py, making it ideal for comparing Python vs JS implementations

Custom Autograd Engine

• Value class tracks scalar operations and computes gradients via reverse-mode autodiff (backprop)
• Implements add, mul, tanh, exp, pow, and composite ops with associated backward functions
• Inspired by Karpathy's micrograd

Miniature GPT Architecture

• Character-level tokenizer: each unique character (+ BOS token) mapped to an integer
• Token embeddings (wte) and positional embeddings (wpe), summed per position
• Single transformer decoder block: multi-head self-attention (4 heads, 16-dim, 16-token context) + feedforward MLP
• RMSNorm (no learnable parameters) applied before each sublayer (pre-norm)
• Adam optimizer hand-rolled from first principles

Interactive Training Playground

• Upload any .txt file or use the bundled names.txt example dataset
• Live loss display during training with periodic inference samples
• Demonstrated on character-level name generation (baby names dataset)

Technical Architecture

Input Text
    │
    ▼
Character-level Tokenizer
    │  (unique chars + BOS → int IDs)
    ▼
Embedding Lookup
    │  wte[token_id] + wpe[position]
    ▼
Transformer Decoder Block
    │  ┌─ RMSNorm
    │  ├─ Multi-Head Self-Attention (Q/K/V projections, causal mask)
    │  ├─ Residual Add
    │  ├─ RMSNorm
    │  ├─ Feedforward MLP (Linear → GELU → Linear)
    │  └─ Residual Add
    ▼
LM Head (unembedding: Linear → logits)
    │
    ▼
Softmax → Cross-Entropy Loss (training) / Sampling (inference)
    │
    ▼
Adam Optimizer Update (custom, scalar autograd)

All operations are implemented as scalar arithmetic using the Value autograd class; no tensor library is used.

Installation & Usage

HuggingFace Space (no install):

https://huggingface.co/spaces/webml-community/microgpt-playground

Node.js / CLI:

# Clone and run directly — no npm install required
git clone https://github.com/xenova/microgpt.js
node microgpt.js

Browser (script tag):

<script type="module">
  import { train, generate } from './microgpt.js';
  // Load text, call train(), then generate()
</script>

Web Worker (non-blocking training):

const worker = new Worker('./microgpt-worker.js');
worker.postMessage({ text: myTextData });
worker.onmessage = (e) => console.log('Loss:', e.data.loss);

Relevance to Claude Code Development

Applications

• Educational reference: Demonstrates the complete GPT pipeline in ~200 lines — useful when building skills that explain transformer internals or LLM concepts
• Browser-hosted agent tools: The pattern of full ML inference running client-side (no server) applies to lightweight Claude Code plugins that need embedded inference
• Algorithm verification: Compare Python and JavaScript implementations side-by-side to validate porting exercises or cross-language agent skills

Patterns Worth Adopting

• Zero-dependency single-file design: Ship complete functionality as one file with no external dependencies — applies to Claude Code skill scripts that need to be portable
• Web Worker pattern for CPU-bound tasks: Offload long-running Claude Code hooks or background skill logic to Web Workers to avoid blocking the main thread in browser-based UIs
• Scalar autograd as teaching tool: When building skills that teach AI concepts, a minimal autograd engine is more pedagogically effective than importing a full framework

Integration Opportunities

• Could power a Claude Code skill that trains a tiny character-level LM on the user's codebase for local completion suggestions (no external API calls)
• Demonstrates how to run any Python ML algorithm in-browser — relevant for Claude Code plugins deployed as HuggingFace Spaces
• The HF Space deployment model (static files + JS) is a replicable pattern for lightweight Claude Code demos

References

• microgpt Playground (HuggingFace Space) (accessed 2026-02-23)
• xenova/microgpt.js (GitHub) (accessed 2026-02-23)
• Karpathy's microgpt.py gist (accessed 2026-02-23)
• webml-community on HuggingFace (accessed 2026-02-23)
• Andrej Karpathy's microGPT Architecture — Complete Guide (accessed 2026-02-23)
• Karpathy's microgpt.py Dissected (accessed 2026-02-23)

Freshness Tracking