Cutile Dsl Ref

cuTile Python DSL Reference

Execution Model

• Tiles are chunks of data loaded from HBM into SMEM (shared memory)
• Operations work on tiles, not individual elements
• TMA (Tensor Memory Accelerator) handles async HBM-SMEM transfers (allow_tma=True)
• Grid of thread blocks: use ct.bid(0/1/2) for block indices
• Compiler generates optimized MLIR → PTX/SASS

Basically cuTile allows to divide data into tiles, and define which blocks (mapped with ct.bid(0/1/2)) will process which tiles. The scheduler/compiler will do the mapping to the actual hardware resources.

Scheduling and Clustering Boundaries

The execution model is intentionally block-level, not thread-level. The official cuTile execution docs state that tile programs describe block-level parallelism, that threads cannot be explicitly identified or manipulated, and that explicit synchronization or communication within a block is not permitted.

Practical consequence: cuTile lets you choose block work assignment and compiler hints, but it does not expose explicit intra-CTA scheduling primitives. There is no public API for thread, warp, or warpgroup identity; no user-visible producer/consumer warp specialization; no explicit barrier choreography; and no public pipeline controls comparable to handwritten CUDA or CUTLASS/CuTe kernels.

For clustering, num_ctas is real, but it is only a cluster-size entry hint. The compiler lowers it to num_cta_in_cga, yet cuTile does not expose a cluster-local programming model such as cluster rank, cluster barrier, DSM primitives, or cluster memory scope. Treat num_ctas as a placement/codegen hint, not as an API for explicit cluster-cooperative algorithms.

This limitation matters most for kernels that are structurally forced into one CTA per SM by register pressure or shared-memory footprint. In that regime, high-end kernels often recover utilization through explicit warpgroup scheduling, fine-grained copy/compute pipelining, or cluster-cooperative work distribution. cuTile cannot express those controls directly, so if profiling shows that one-block-per-SM residency is the root bottleneck and the missing win requires explicit scheduling, a lower-level DSL or CUDA path is usually the correct next step.

Core API Table

Key Constraints

1. No ct.softmax() — implement manually with online running statistics
2. ct.mma requires compatible shapes for tensor cores (M/N/K multiples of 16/8)
3. Latency hints are suggestions — validated with profiling, not guaranteed
4. ct.load order param — axis permutation for transposed loads (e.g., 'F', (0, 2, 1))
5. ConstInt params — tile sizes must be compile-time constants (passed as ConstInt)
6. Grid up to 3D — (x, y, z) dimensions
7. No explicit intra-CTA control — no thread IDs, warp IDs, warpgroup IDs, or in-block barrier choreography
8. No public cluster programming model — num_ctas selects CGA size, but there is no exposed cluster rank/barrier/DSM API

Compiler Hints

@ct.kernel(num_ctas=2, occupancy=4)
def my_kernel(...):

Persistent Kernel Grid Formula

When combining persistent kernels with CGA clusters, account for both:

# num_ctas affects available cluster count (not specific to persistent kernels)
grid_size = min(NUM_SMS // num_ctas, num_tiles) * occupancy

@ct.kernel(num_ctas=num_ctas_val)
def persistent_kernel(...):
    bid = ct.bid(0)
    num_programs = ct.num_blocks(0)
    for tile_id in range(bid, num_tiles, num_programs):
        # Process tile

Common Patterns

Online Softmax (FlashAttention-style)

m_i = ct.full((Bm,), -math.inf, dtype=ct.float32)    # running max
l_i = ct.full((Bm,), 1.0, dtype=ct.float32)           # running sum
acc = ct.zeros((Bm, D), dtype=ct.float32)              # accumulator

for kv_idx in range(0, T, Bn):
    q = ct.load(Q, ...)
    k = ct.load(K, ...)
    qk = ct.mma(q, k, ct.zeros((Bm, Bn), dtype=ct.float32))
    # Add positional scores if needed
    m_ij = ct.max(qk, axis=-1)
    m_new = ct.maximum(m_i, m_ij)
    alpha = ct.exp2(m_i - m_new)
    p = ct.exp2(qk - m_new[:, None])
    l_ij = ct.sum(p, axis=-1)
    l_i = l_i * alpha + l_ij
    acc = acc * alpha[:, None]
    v = ct.load(V, ...)
    acc = ct.mma(p, v, acc)
    m_i = m_new

# Normalize
acc = acc / l_i[:, None]

Head Grouping (Bh > 1)

# Load query for Bh heads at once
q = ct.load(Qc, shape=(1, k, Bh, Bm), ...)
q = ct.reshape(q, (k, Bh * Bm))
# Z tile shared across all Bh heads (loaded ONCE per block)
z = ct.load(Z, shape=(1, Bn, k), ...)
qk = ct.mma(q, z, acc)  # (Bh*Bm, Bn) — all heads at once

Split-KV Block Assignment

split_idx = ct.bid(0)       # which split (latent or decompressed)
path_id = ct.bid(1)         # batch * head_groups * query_tiles
is_latent = split_idx < C_o
if is_latent:
    start = split_idx * L_o
    # process latent segment
else:
    start = (split_idx - C_o) * L_n
    # process decompressed segment

Official Documentation Index

Load these files on demand when you need detailed API information:

Example Kernels

Study these for complete implementation patterns:

Detailed References

• API deep-dives: See references/api-index.md for organized links to official cuTile documentation sections
• Scheduling boundary evidence: See docs/cutile-dsl/execution.md and docs/cutile-dsl/performance.md for the public model, and docs/kernels/flash-mla.md for the concrete FlashMLA consequence when the kernel is pinned to one CTA per SM