Torch Performance Optimization
Read docs/torch-performance-optimization.md completely before inspecting or editing a Torch path. Also read ../performance-optimization/SKILL.md, its canonical performance guide, and ../albucore-benchmarks/SKILL.md completely.
Workflow
- Establish a correctness baseline and benchmark the existing public path. State container, layout, dtype, shape, parameter, stride, thread, and allocation contracts.
- Audit unnecessary Python work, full-volume passes, conversions, materializations, and output repairs before choosing a Torch operator.
- Profile only to discover candidates. Benchmark every viable implementation end to end, including NumPy↔Torch bridges and layout conversion.
- Compare NumPy, OpenCV, NumKong, StringZilla, and Torch where they share semantics. Treat fused operators, layout,
channels_last_3d, and in-place reuse as hypotheses with their own correctness and performance matrices.
- Select a route or threshold only from stable public-path evidence. Preserve rejected candidates in the benchmark report when they clarify a boundary.
- Add correctness tests for each accepted route and update the benchmark evidence, public contract, and canonical guide when a reusable rule or limitation is discovered.
Albucore Contract
- Torch is a required dependency. Assume it is already imported for benchmarks.
- Current public Tensor routes are eager CPU paths that do not record autograd inside the primitive and do not use
torch.compile. Do not add device routes, graph-preserving primitive fallbacks, compilation, or their benchmark candidates.
- Tensor layouts are explicit and independent of NumPy layouts. Do not infer
NCHW versus CDHW from shape sizes.
- Caller-prevalidated 3D routers leave validation outside the hot path. Do not add it back while optimizing.
- A Tensor route must preserve the documented container, layout, border, interpolation, rounding, mutation, and aliasing behavior. Its dtype contract must distinguish supported dtype preservation from any explicit fallback conversion.
Required Handoff
Report the full-path baseline, selected route, removed work, allocation and copy changes, all viable backends and Torch-specific candidates considered, thread and benchmark matrix, correctness/memory evidence, regressions, rejected regions, and remaining follow-ups.
1---2name: torch-performance-optimization3description: Optimize or review eager CPU-only Albucore PyTorch runtime paths with benchmark-backed decisions. Use when adding or changing Torch CPU kernels, Tensor/NumPy bridges, Torch backend routing, tensor layouts, allocations, threading, profiling, memory-format candidates, or Torch performance benchmarks.4---56# Torch Performance Optimization78Read [`docs/torch-performance-optimization.md`](../../../docs/torch-performance-optimization.md) completely before inspecting or editing a Torch path. Also read `../performance-optimization/SKILL.md`, its canonical performance guide, and `../albucore-benchmarks/SKILL.md` completely.910## Workflow11121. Establish a correctness baseline and benchmark the existing public path. State container, layout, dtype, shape, parameter, stride, thread, and allocation contracts.132. Audit unnecessary Python work, full-volume passes, conversions, materializations, and output repairs before choosing a Torch operator.143. Profile only to discover candidates. Benchmark every viable implementation end to end, including NumPy↔Torch bridges and layout conversion.154. Compare NumPy, OpenCV, NumKong, StringZilla, and Torch where they share semantics. Treat fused operators, layout, `channels_last_3d`, and in-place reuse as hypotheses with their own correctness and performance matrices.165. Select a route or threshold only from stable public-path evidence. Preserve rejected candidates in the benchmark report when they clarify a boundary.176. Add correctness tests for each accepted route and update the benchmark evidence, public contract, and canonical guide when a reusable rule or limitation is discovered.1819## Albucore Contract2021- Torch is a required dependency. Assume it is already imported for benchmarks.22- Current public Tensor routes are eager CPU paths that do not record autograd inside the primitive and do not use `torch.compile`. Do not add device routes, graph-preserving primitive fallbacks, compilation, or their benchmark candidates.23- Tensor layouts are explicit and independent of NumPy layouts. Do not infer `NCHW` versus `CDHW` from shape sizes.24- Caller-prevalidated 3D routers leave validation outside the hot path. Do not add it back while optimizing.25- A Tensor route must preserve the documented container, layout, border, interpolation, rounding, mutation, and aliasing behavior. Its dtype contract must distinguish supported dtype preservation from any explicit fallback conversion.2627## Required Handoff2829Report the full-path baseline, selected route, removed work, allocation and copy changes, all viable backends and Torch-specific candidates considered, thread and benchmark matrix, correctness/memory evidence, regressions, rejected regions, and remaining follow-ups.
albumentations-team