Jit Gui Expert

Just-in-Time GUI Expert

Use this skill when designing interfaces for AI-native products, voice-first assistants, or any system where UI should be generated dynamically rather than pre-designed statically.

1. What Is Just-in-Time GUI?

Definition: A UX paradigm where interfaces are generated dynamically, on-demand, and ephemerally — existing only as long as needed. AI/LLMs assemble UI in real-time based on user context and intent, rather than using pre-designed static screens.

Coined by: Andrew Sims (Signal Path) — "emerging UX patterns for AI-native products that trade static screens and flows for UI assembled on the fly."

Core philosophy: The interface is invisible by default and surfaces only when context demands it — then disappears when its purpose is served.

Traditional GUI vs JIT GUI

2. Core Principles

1. Context-Dependency — UI depends on accurate context (what user is doing, history, priorities)
2. Ambiguity & Adaptation — Design for uncertainty, not fixed flows
3. Character-Based Design — AI agents as "characters with personalities and logic," not just backends
4. Outcome-Oriented — Define user goals/constraints for AI to operate within (not discrete design elements)
5. Stateless Paradigm — All interface state derives from external context (user intent, app state, environment)
6. Calm Technology — Interface moves between periphery and center of attention naturally (Weiser & Brown, 1995)
7. Zero Inventory — No unused features sitting in the codebase; generate what's needed, discard the rest

3. Related Paradigms

Ambient Computing / Zero UI

• Eliminates traditional screens/buttons; relies on voice, gesture, predictive automation
• Technology complements the human environment seamlessly, not the other way around

Calm Technology (Weiser & Brown, 1995)

• Technology should serve, not demand attention
• Interfaces fade to background while remaining available
• Move between periphery and center of attention naturally

Ephemeral Interfaces

• Generated on-the-fly, exist temporarily, auto-dismiss when purpose is served
• Voice interfaces are naturally ephemeral (audio is fleeting)
• "Ephemeral apps": AI field-codes a tiny temporary tool for the task, then it disappears

AX (Agentic Experience) — Replacing UX

• Jakob Nielsen 2026: Agentic design becomes default expectation
• Interface disappears when not needed; reappears contextually
• AI acts as silent co-designer adjusting based on user behavior
• Success metric: "tasks completed autonomously" not "tokens generated"

4. Three Implementation Patterns

Pattern 1: Static Generative UI (High Control, Low Freedom)

Pre-built component library → Agent selects which + passes data → Frontend renders

• Safest (security), most consistent
• Agent only controls when/which, not how
• Best for: production approval flows, dashboards, forms with fixed layouts

Pattern 2: Declarative Generative UI (Shared Control) — RECOMMENDED

Agent returns structured spec (JSON/JSONL) → Frontend interprets → Renders consistently

• Pre-approved components only — security by design
• Cross-platform: SwiftUI, Flutter, React from same spec
• LLM-friendly: flat, incrementally streamable
• Best for: most production use cases
• Protocols: A2UI, Open-JSON-UI

Pattern 3: Open-Ended Generative UI (Low Control, High Freedom)

Agent returns complete HTML/iframe → Frontend acts as container

• Maximum flexibility, highest risk (XSS, performance, inconsistency)
• Best for: experimental features, sandboxed tools only

Data Flow (All Patterns)

User Input → LLM → JSON/Spec Generation → Frontend Rendering → User Interaction
                     ↓
                  Tool Calls → Data Context

5. 2026 Protocol Stack

Full stack: A2A → MCP (tools) → AG-UI (runtime) → A2UI/Open-JSON-UI (payload)

A2UI (Google's Standard) — Deep Dive

Format: JSON Lines (JSONL) — each line is a complete message.

Why A2UI wins for voice-first:

• Security-first: declarative data, NOT executable code
• Framework-agnostic: SwiftUI, Flutter, React, Angular from same spec
• LLM-friendly: flat list of components with ID references = easy for LLMs to generate incrementally
• Streaming: each JSONL line can be processed as it arrives

Adopted by: Google, LangChain, AWS, Microsoft, Mastra, PydanticAI

Example:

{"component": "card", "id": "c1", "title": "Meeting in 15 min"}
{"component": "text", "id": "t1", "content": "Team standup with 4 attendees"}
{"component": "button", "id": "b1", "label": "Join", "onclick": "action:join_meeting"}
{"component": "button", "id": "b2", "label": "Snooze 5m", "onclick": "action:snooze"}

Client implements:

1. Maintains a "catalog" of pre-approved components (Card, Button, TextField, etc.)
2. Agent can only request rendering of trusted, predefined components
3. Client maps abstract descriptions to native widgets (SwiftUI views, React components)
4. Unknown components are safely ignored

6. Voice-First JIT GUI Patterns

Architecture for Voice + Ephemeral UI

User speaks → Whisper transcribes → Manager LLM reasons
                                         ↓
                              ┌─────────────────────────┐
                              │ LLM decides output mode: │
                              ├─────────────────────────┤
                              │ Voice only → TTS         │
                              │ Voice + Card → TTS + UI  │
                              │ Approval → Button card   │
                              │ Data → Visual widget     │
                              └─────────────────────────┘
                                         ↓
                              SSE → Client renders ephemerally
                              Auto-dismiss after timeout/action

Notification-as-Interface

Instead of opening windows/apps, use cards as the primary visual surface:

Ephemeral Lifecycle

1. Appear on SSE event from server
2. Live for duration of interaction
3. Auto-dismiss after: task completion | user swipe | timeout (30s) | voice command ("dismiss")
4. Never persist — no saved state, no history

Multimodal Seamlessness

• Voice triggers request (fast, natural)
• Visual confirms action (reassurance, data display)
• Click/tap for fine control (precision, selection)
• No modality treated as separate — best one naturally emerges per context

7. Key Design Rules

The Calm Technology Test

Before adding any visual UI, ask:

1. Can this be voice-only? If yes, don't add UI.
2. Does the user need to SEE something? (data, options, confirmation)
3. Will this demand attention or complement it?
4. Does it auto-dismiss, or does user need to manage it?

8. Thought Leaders (Quick Reference)

• Andrew Sims (Signal Path) — Coined "Just-in-Time Interfaces"; AI agents as characters, not services
• Linus Lee (Thrive Capital) — Moving from "language space" to "meaning space"; pinch-to-summarize, dial-for-formality
• Amelia Wattenberger — Chatbots lack flow state & sensory richness; JIT GUI needs multimodal feedback
• Jakob Nielsen — 2026: agentic design default; interface disappears when unneeded, reappears contextually
• Yaniv Leviathan (Google) — Proved LLMs generate high-quality UIs; released PAGEN dataset

9. For Extended Reference

See reference.md in this skill directory for:

• Full frameworks & libraries table (Vercel AI SDK, CopilotKit, assistant-ui, etc.)
• Academic papers with citations (5 key papers)
• All source URLs organized by category
• Real-world implementation details (Dynamic Island, Google GenUI, MCP Apps)
• Code examples (A2UI JSONL, Vercel streamUI, JSON Schema forms)