Cuda Attention Kernel Patterns

Patterns and pitfalls for the ONNX domain Attention operator (opset 23/24) CUDA implementation. Use when modifying the dispatch cascade in core/providers/cuda/llm/attention.cc, writing mask/bias CUDA kernels, debugging attention test routing, or adding features to the ONNX Attention op. NOT for contrib domain MultiHeadAttention/GroupQueryAttention.

Microsoft Updated 195k repo stars

File contents

microsoft/onnxruntime commit 6d916a73e8

Frequently asked questions

npx get-skill add microsoft/cuda-attention-kernel-patterns