# High Performance Computing

> Performance optimization. Vectorization, memory layout, GPU acceleration, profiling, parallelism.

- Skill: `aselimc/high-performance-computing` (Agent Skill)
- Install (CLI): `npx skillmds@latest add aselimc/high-performance-computing`
- Raw SKILL.md: https://api.skillmd.com/api/skills/aselimc/high-performance-computing/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: aselimc (https://skillmd.com/u/aselimc)
- Updated: 2026-09-10
- Page: https://skillmd.com/skills/aselimc/high-performance-computing

---


# High-Performance Computing

## Profiling First
```bash
# Python
python -m cProfile -o prof.out script.py
py-spy record -o profile.svg -- python script.py
# GPU
nsight systems profile python train.py
```

## Vectorization
- Replace Python loops with NumPy/JAX operations
- Use `jax.jit` for JIT compilation of pure functions
- Batch operations: process N items at once, not one-by-one

## Memory Layout
- AoS (Array of Structs) vs SoA (Struct of Arrays): SoA for SIMD/GPU
- Contiguous memory: `np.ascontiguousarray()`, `tensor.contiguous()`
- Avoid unnecessary copies: use views, in-place operations

## GPU Acceleration
```python
import jax
@jax.jit
def compute(x): return jax.numpy.dot(x, x.T)
```
Or Numba for CUDA kernels, CuPy for drop-in NumPy replacement.

## Parallelism
- `multiprocessing.Pool` for CPU-bound
- `joblib.Parallel` for embarrassingly parallel
- `torch.distributed` for multi-GPU training

## Key Libraries
JAX, NumPy, Numba, CuPy, line_profiler