# Typegpu

> TypeGPU and raw WebGPU adapter patterns for FrameVideo. Use when creating GPU-rendered compositions with TypeGPU, raw WebGPU, WGSL fragment shaders, compute pipelines, liquid glass effects, particle systems, or any canvas layer driven by navigator.gpu that responds to FrameVideo fv-seek events.

- Skill: `chanjing-ai/typegpu` (Agent Skill)
- Install (CLI): `npx skillmds@latest add chanjing-ai/typegpu`
- Raw SKILL.md: https://api.skillmd.com/api/skills/chanjing-ai/typegpu/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: chanjing-ai (https://skillmd.com/u/chanjing-ai)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/chanjing-ai/typegpu

---


# TypeGPU / WebGPU for FrameVideo

## When To Use

Use TypeGPU/WebGPU for:

- **GPU compute shaders** — particle systems, physics simulations
- **Custom WGSL shaders** — liquid glass, metaballs, ray marching
- **High-performance effects** — complex fragment shaders, post-processing
- **Advanced visual effects** — noise fields, reaction-diffusion, fluid simulations
- **Compute-heavy visuals** — when Three.js is too high-level

## Do NOT Use

Avoid TypeGPU/WebGPU for:

- **Simple 3D scenes** — use `three` (easier and more mature)
- **2D animations** — use `gsap` (much simpler)
- **Standard effects** — WebGPU has limited browser support, use WebGL alternatives
- **Text/UI animation** — use HTML + GSAP (better rendering)

---

## Quick Start

Basic WebGPU fragment shader in FrameVideo:

```html
<canvas id="gpu-layer" width="1920" height="1080"></canvas>

<script>
  (async () => {
    if (!navigator.gpu) return;
    
    const adapter = await navigator.gpu.requestAdapter();
    const device = await adapter.requestDevice();
    const canvas = document.getElementById("gpu-layer");
    const ctx = canvas.getContext("webgpu");
    
    ctx.configure({ 
      device, 
      format: navigator.gpu.getPreferredCanvasFormat() 
    });

    // Create time uniform buffer
    const timeUniform = new Float32Array([0]);
    const timeBuf = device.createBuffer({
      size: 16,
      usage: GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST,
    });

    // Listen for FrameVideo seek events
    window.addEventListener("fv-seek", async (event) => {
      const time = event.detail.time;
      
      // Update time uniform
      timeUniform[0] = time;
      device.queue.writeBuffer(timeBuf, 0, timeUniform);
      
      // Render your pipeline...
      // [render pass code here]
      
      // IMPORTANT: flush GPU work before frame capture
      await device.queue.onSubmittedWorkDone();
    });
  })();
</script>
```

**Key points:**
1. Listen for `fv-seek` event
2. Update uniforms with `event.detail.time`
3. Call `await device.queue.onSubmittedWorkDone()` before frame capture
4. Guard for WebGPU availability

---

## Contract

- Initialize WebGPU asynchronously (`await navigator.gpu.requestAdapter()`), but register all GSAP tweens **synchronously** — before any `await`. The FrameVideo player reads the timeline immediately at page load.
- Render from FrameVideo time, not `performance.now()`.
- Listen for the `fv-seek` event and re-render at exactly that time.
- Guard against environments where WebGPU is unavailable — the adapter does not check for you.
- For video renders, call `await device.queue.onSubmittedWorkDone()` after submitting GPU work to ensure the canvas is flushed before the frame is captured.

The adapter sets `window.__fvTypegpuTime` and dispatches `new CustomEvent("fv-seek", { detail: { time } })` on each seek.

## Basic Pattern

```html
<canvas id="gpu-layer"></canvas>
<script>
  (async () => {
    if (!navigator.gpu) return;
    const adapter = await navigator.gpu.requestAdapter();
    if (!adapter) return;
    const device = await adapter.requestDevice();
    const canvas = document.getElementById("gpu-layer");
    canvas.width = 1920;
    canvas.height = 1080;
    const ctx = canvas.getContext("webgpu");
    const fmt = navigator.gpu.getPreferredCanvasFormat();
    ctx.configure({ device, format: fmt, alphaMode: "opaque" });

    // Build your pipeline, buffers, bind groups...
    const timeUniform = new Float32Array([0]);
    const timeBuf = device.createBuffer({
      size: 16,
      usage: GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST,
    });

    function render(t) {
      timeUniform[0] = t;
      device.queue.writeBuffer(timeBuf, 0, timeUniform);
      const enc = device.createCommandEncoder();
      const pass = enc.beginRenderPass({
        colorAttachments: [
          {
            view: ctx.getCurrentTexture().createView(),
            loadOp: "clear",
            clearValue: { r: 0, g: 0, b: 0, a: 1 },
            storeOp: "store",
          },
        ],
      });
      pass.setPipeline(pipeline);
      pass.setBindGroup(0, bindGroup);
      pass.draw(3);
      pass.end();
      device.queue.submit([enc.finish()]);
    }

    render(0);
    window.addEventListener("fv-seek", (e) => render(e.detail.time));
  })();
</script>
```

## Timeline Registration

GSAP tweens that drive text, captions, or HTML elements must be registered **synchronously** — before any `await`:

```js
const tl = gsap.timeline({ paused: true });

// Caption tweens: synchronous, added before WebGPU init
gsap.set(".cap", { opacity: 0 });
tl.to("#cap-1", { opacity: 1, duration: 0.3 }, 1.0);
tl.to("#cap-1", { opacity: 0, duration: 0.2 }, 3.5);

window.__timelines["my-comp"] = tl;

// GPU-dependent tweens can go inside the async IIFE
(async () => {
  // ... WebGPU init ...
  const proxy = { value: 0 };
  tl.to(proxy, { value: 1, duration: 2, onUpdate: render }, 0.5);
})();
```

## Video-Backed Effects (Liquid Glass, Distortion)

To use a `<video>` as the GPU input texture:

```js
const videoEl = document.getElementById("aroll");

// Wait for video metadata before creating the texture
await new Promise((r) => {
  if (videoEl.readyState >= 1) r();
  else videoEl.addEventListener("loadedmetadata", r, { once: true });
});

// Create texture at the video's NATIVE resolution
const vw = videoEl.videoWidth,
  vh = videoEl.videoHeight;
const bgTex = device.createTexture({
  size: [vw, vh],
  format: "rgba8unorm",
  usage:
    GPUTextureUsage.COPY_DST | GPUTextureUsage.TEXTURE_BINDING | GPUTextureUsage.RENDER_ATTACHMENT,
});

function render(t) {
  try {
    device.queue.copyExternalImageToTexture({ source: videoEl }, { texture: bgTex }, [vw, vh]);
  } catch (_) {
    /* frame not decoded yet */
  }
  // ... draw ...
}
```

**Render-mode caveat:** headless Chrome may fail `copyExternalImageToTexture` for video elements. For production renders, pre-extract key frames via FFmpeg as PNGs and load them as image textures instead.

## Frosted Blur via Downsample Pass

A single-pass Gaussian kernel is too weak for glass-like frosted blur. Use a two-pass approach:

1. **Pass 1 — Downsample:** render the full-res texture to a small texture (1/6 resolution). Bilinear filtering during the downsample naturally averages pixels.
2. **Pass 2 — Glass composite:** sample the small texture for the frosted interior (bilinear upscale = heavy smooth blur) and the full-res texture for sharp areas and chromatic refraction.

This matches TypeGPU's `textureSampleBias` mip-level approach without generating mipmaps.

## Transparent vs Opaque Canvas

- **`alphaMode: 'opaque'`** — the GPU canvas renders the full frame (video + effect). Use when the GPU pipeline handles all visual content.
- **`alphaMode: 'premultiplied'`** — the GPU canvas is transparent where alpha = 0, letting HTML elements below show through. Use for overlays (particles, path animations) on top of a regular `<video>` element.

## WGSL Full-Screen Triangle

The standard vertex shader for full-screen effects (no vertex buffer needed):

```wgsl
struct Vo { @builtin(position) pos: vec4f, @location(0) uv: vec2f }

@vertex fn vs(@builtin(vertex_index) vi: u32) -> Vo {
  let ps = array<vec2f, 3>(vec2f(-1., -1.), vec2f(3., -1.), vec2f(-1., 3.));
  let ts = array<vec2f, 3>(vec2f(0., 1.), vec2f(2., 1.), vec2f(0., -1.));
  return Vo(vec4f(ps[vi], 0., 1.), ts[vi]);
}
```

Draw with `pass.draw(3)` — one triangle that covers the viewport.

## Rounded-Rect SDF (Liquid Glass Pill)

```wgsl
fn sdf_box(p: vec2f, half_size: vec2f, corner_radius: f32) -> f32 {
  let d = abs(p) - half_size + vec2f(corner_radius);
  return length(max(d, vec2f(0.))) + min(max(d.x, d.y), 0.) - corner_radius;
}
```

Use this to define inside/ring/outside zones for glass effects. Negative values are inside the shape.

## Deterministic Rendering

- No `Math.random()` — use a seeded PRNG.
- No `requestAnimationFrame` for the render loop — render only in response to `fv-seek`.
- No `performance.now()` for animation time — read `window.__fvTypegpuTime` or `e.detail.time`.
- After GPU submit, call `await device.queue.onSubmittedWorkDone()` for render-mode frame capture.