Results for “attention”
6 skills
orchestra-research Optimizing Attention Flash
Optimizes transformer attention with Flash Attention for 2-4x speedup and 10-20x memory reduction. Supports PyTorch native SDPA, flash-attn library, H100 FP8, and sliding window attention.
10.4k · bundle
nvidia Tao Train Ocrnet
Trains, evaluates, exports, prunes, quantizes, retrains, and runs inference for TAO OCRNet models for scene text recognition from cropped text-region images, supporting CTC and attention-based decoders.
2.2k · bundle
orchestra-research Rwkv Architecture
Use RWKV, a linear-time RNN-Transformer hybrid, for efficient long-context inference and training with constant memory usage.
10.4k · bundle
More results
orchestra-research Deepspeed
Provides expert guidance for distributed training with DeepSpeed, covering ZeRO optimization stages, pipeline parallelism, FP16/BF16/FP8, 1-bit Adam, and sparse attention.
10.4k · bundle
nvidia Nemo Mbridge Perf Cuda Graphs
Validate and use CUDA graph capture in Megatron Bridge, including local full-iteration graphs and Transformer Engine scoped graphs for attention, MLP, and MoE modules.
2.2k · bundle
nvidia Tao Train Deformable Detr
Train, evaluate, export, quantize, and run inference for a Deformable DETR 2D object detection model using TAO, with deformable attention for efficient multi-scale feature processing.
2.2k · bundle