# Gpu Opt Tier4 Advanced Scheduling

> Applies Split-K, persistent kernels, Stream-K/Stream-K++, warp specialization, and multi-GPU fused schedules. Use when simple tiling leaves tails, load imbalance, or excessive kernel launch overhead on large GPUs.

- Skill: `zjli2013/gpu-opt-tier4-advanced-scheduling` (Agent Skill)
- Install (CLI): `npx skillmds@latest add zjli2013/gpu-opt-tier4-advanced-scheduling`
- Raw SKILL.md: https://api.skillmd.com/api/skills/zjli2013/gpu-opt-tier4-advanced-scheduling/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: ZJLi2013 (https://skillmd.com/u/zjli2013)
- Updated: 2026-09-22
- Page: https://skillmd.com/skills/zjli2013/gpu-opt-tier4-advanced-scheduling

---


# Tier 4: Advanced Scheduling

**AutoKernel Tier 4** addresses **parallelism structure** beyond per-block tiling: how work maps to SMs over time and across chips.

## Split-K with reduction

- **Idea**: Multiple blocks partial-sum along K; **atomic** or **second-phase reduction** to finalize C.
- **Use when**: Increasing `BLOCK_K` alone overspills SMEM/regs or hurts occupancy, but more **K-parallelism** improves utilization.
- **Cost**: Extra **HBM traffic** for partials unless reduction is fused; watch **atomics** on small tiles.

## Persistent kernels

- **Idea**: Launch **~#SMs** blocks (or similar) that **loop internally** over many output tiles, reusing registers/SMEM across tiles.
- **Benefits**: Amortizes **launch latency**, improves **L2 locality** with controlled traversal order.
- **Risks**: Complex **grid-stride** indexing; harder debugging. Pair with **ordered tile scheduling** for cache affinity.

## Stream-K / Stream-K++

- **Problem**: Classic 2D tiling leaves **tail imbalance** (last waves idle).
- **Idea**: **Stream** fractional K contributions across blocks so SMs stay busy; **Stream-K++** refines reduction and scheduling for fewer barriers.
- **When**: Large rectangular GEMMs on **high SM-count** GPUs where occupancy is uneven.

## Warp specialization (Hopper-class)

- **Pattern**: **Producer warps** (TMA / async copy / transport) vs **consumer warps** (WGMMA / math).
- **Goal**: Overlap **bulk memory movement** with **tensor-core compute** without starving either side.
- **When**: Kernels already use **TMA/WGMMA** pipelines; not a first step for simple Triton dot kernels.

## Multi-GPU: compute + communication fusion

- **Overlap**: **NCCL/RCCL** collectives scheduled alongside **independent compute** on separate streams (or fused schedules in framework runtimes).
- **Fusion patterns**: **All-reduce + gradient GEMM** pipelining, **tensor-parallel** shards with **next-layer prefetch**.
- **Watch**: **Bubble** minimization dominates; profile both **network** and **GPU** timelines.

## Picking a strategy (quick matrix)

| Symptom | Try first | Watch for |
|---------|-----------|-----------|
| K too large for one tile’s SMEM | **Split-K** | Partial traffic + atomics |
| Huge grid, tiny per-kernel time | **Persistent** | Correctness of tile iterator |
| Classic 2D tail idle on big GPU | **Stream-K** | Reduction complexity |
| Hopper + TMA in play | **Warp specialization** | Producer/consumer deadlock |
| TP/PP pipeline bubbles | **Comm/compute overlap** | Ordering / race bugs |

## Interaction with AutoKernel Tier 6

- Some playbooks treat **tooling/automation** (search, DSL lowering) as a final tier; scheduling choices here should still be **expressed as reproducible config knobs** for autotuners.

## Agent Instructions

1. Only adopt Tier 4 after **Tier 1–3** are sane; confirm imbalance or launch overhead with a **timeline profiler** (NSYS/rocprof traces).
2. For **tail-heavy** matmuls, prototype **Stream-K** or **persistent** variants against a **Split-K** baseline; compare **effective TFLOPS** and **code complexity**.
3. On **Hopper**, if using advanced APIs, evaluate **warp-specialized** pipelines vs monolithic blocks using profiling counters for **TMA vs math overlap**.
4. For **multi-GPU**, map **critical path** (comm vs compute) and prefer **fused/overlapped** schedules where safe.
5. Align documentation and tuning notes with **AutoKernel Tier 5** (vendor-specific caps) before shipping production configs.