# Numa Programming > Use when detecting NUMA topology, binding processes with numactl, using the libnuma API, building NUMA-aware data structures, or measuring remote memory access penalties. - Skill: `outlinedriven-odin-claude-plugin/numa-programming` (Agent Skill, multi-file: 2 files) - Install (CLI): `npx skillmds@latest add outlinedriven-odin-claude-plugin/numa-programming` - Raw SKILL.md: https://api.skillmd.com/api/skills/outlinedriven-odin-claude-plugin/numa-programming/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Integrations & APIs - Author: OutlineDriven (https://skillmd.com/u/outlinedriven-odin-claude-plugin) - Updated: 2026-09-10 - Page: https://skillmd.com/skills/outlinedriven-odin-claude-plugin/numa-programming --- # NUMA programming ## Contract | Field | Bound contract | |---|---| | Trigger | NUMA-aware programming or debugging on multi-socket Linux systems. | | Authority | Read-only. No source or remote mutation. Chat output only. | | Side effect | Emits a structured guidance report to chat. | | Done | The report shows the NUMA topology, the right `numactl` binding, libnuma API usage, a per-node data structure pattern, remote access diagnosis, and a decision tree. | ## Inputs 1. **Target system** (required): a multi-socket Linux host or VM with NUMA exposed. 2. **Workload symptom** (optional): poor scaling, memory bandwidth saturation, remote access, or unexpected page migration. 3. **Tools available** (optional): `numactl`, `numactl-dev`, `libnuma-dev`, `perf`, `lstopo`. ## Procedure 1. **Detect the NUMA topology.** Run `numactl --hardware`, `lstopo --of console`, and read `/sys/devices/system/node/node*/meminfo` and `node*/cpulist`. A distance value of 10 usually means local access; larger values indicate remote access. Done when: the node count, CPU lists, memory sizes, and distance matrix are reported. Typical output from `numactl --hardware`: ``` available: 2 nodes (0-1) node 0 cpus: 0-15 node 0 size: 65536 MB node 1 cpus: 16-31 node 1 size: 65536 MB node distances: node 0 1 0: 10 21 1: 21 10 ``` 2. **Bind a process with `numactl`.** Use `--cpunodebind` and `--membind` for strict local placement, `--interleave=all` to spread memory, `--preferred` for a preferred node with fallback, and `--show` to inspect the current policy. Done when: the binding command and policy are selected. ```bash numactl --cpunodebind=0 --membind=0 ./myapp numactl --interleave=all ./myapp numactl --preferred=0 ./myapp numactl --show ``` 3. **Use the libnuma API.** Call `numa_available`, `numa_node_of_cpu`, `numa_alloc_onnode`, `numa_alloc_local`, `mbind`, `set_mempolicy`, and `move_pages`. Compile with `-lnuma`. Done when: the C example and API table are shown. ```c #include #include #include int main(void) { if (numa_available() < 0) { fprintf(stderr, "NUMA not available\n"); return 1; } int node = numa_node_of_cpu(0); printf("CPU 0 on node %d\n", node); size_t size = 1024 * 1024 * 1024; void *mem = numa_alloc_onnode(size, 0); if (!mem) return 1; unsigned long nodemask = 1UL << 0; mbind(mem, size, MPOL_BIND, &nodemask, sizeof(nodemask) * 8, 0); numa_free(mem, size); return 0; } ``` ```bash gcc -o numa_test numa_test.c -lnuma ``` | API | Purpose | |---|---| | `numa_alloc_onnode` | Allocate on a specific node | | `numa_alloc_local` | Allocate on the current CPU node | | `mbind` | Set policy on an existing mapping | | `set_mempolicy` | Default policy for later allocations | | `move_pages` | Migrate pages to a target node | 4. **Build NUMA-aware data structures.** Create a per-node pool and allocate from the node of the current CPU. Pin threads to cores on the same node. Done when: the per-node allocation pattern is shown. ```c #include #include #define BLOCK_SIZE 64 #define MAX_NODES 8 struct per_node_pool { void *free_list[MAX_NODES]; int node_count; }; void *pool_alloc_numa(struct per_node_pool *p) { int node = numa_node_of_cpu(sched_getcpu()); void *blk = p->free_list[node]; if (blk) { p->free_list[node] = *(void **)blk; return blk; } return numa_alloc_onnode(BLOCK_SIZE, node); } ``` 5. **Align thread affinity.** Bind threads to CPUs on a node with `pthread_setaffinity_np` or `numactl --cpunodebind`. Match memory allocation to the same node. Done when: the affinity and memory binding match. ```c #include #include cpu_set_t cpuset; CPU_ZERO(&cpuset); CPU_SET(target_cpu, &cpuset); pthread_setaffinity_np(pthread_self(), sizeof(cpuset), &cpuset); ``` 6. **Diagnose remote access.** Run `perf stat -e cache-misses,cache-references,node-load-misses` with CPU and memory bindings on different nodes. Compare against local-local binding. Use `node-loads` and `node-load-misses` if the kernel exposes them. Done when: the remote access signature is identified. ```bash perf stat -e cache-misses,cache-references,node-load-misses \ numactl --cpunodebind=0 --membind=1 ./myapp perf stat numactl --cpunodebind=0 --membind=0 ./myapp perf stat -e node-loads,node-load-misses,node-stores ./myapp ``` 7. **Measure the remote access penalty.** Touch a large buffer on the local node and then on a remote node. Time the loops. Remote access can be 1.5x to 3x slower than local access on recent multi-socket x86/AMD64 systems with QPI/UPI/Infinity Fabric, depending on the workload and interconnect. Done when: the measurement method is given. ```c clock_t start = clock(); for (size_t i = 0; i < size; i += 4096) sum += ((char *)mem)[i]; ``` 8. **Visualize with `lstopo`.** Run `lstopo` for a graphical view, `lstopo --of ascii` for text, or `lstopo file.png` for an image. Done when: the visualization command is selected. ```bash lstopo lstopo --of ascii lstopo file.png ``` 9. **Walk the decision tree.** Check topology, verify CPU and memory are on the same node, measure `node-load-misses`, then choose `numactl --membind=local`, per-node partitioning, or memory bandwidth reduction. Done when: the next action is named. ``` Poor scaling on multi-socket? ├── Check numactl --hardware ├── Verify thread and memory on the same node ├── perf stat node-load-misses ├── Remote misses high? │ ├── numactl --membind=local │ └── Per-node data partitioning └── Still slow: memory bandwidth bound; reduce sharing ``` ## Failure and recovery | Failure class | Behavior | |---|---| | OOM on one node despite free RAM elsewhere | `MPOL_BIND` is too strict. Use `--preferred` or `--interleave`. | | 2x slower after scaling threads | Threads access remote memory. Bind memory to the same node as the CPU. | | Inconsistent benchmark results | The OS migrated pages. Use `mbind` with `MPOL_BIND` or lock pages with `mlock` if needed. | | DPDK NIC on the wrong socket | The PCI device is far from the CPU. Use `lstopo` and bind EAL to the local socket. | | libnuma not found | The `libnuma-dev` package is missing. Install it with the system package manager. | | First-touch policy surprise | Allocation happened on node 0 but the thread ran on node 1. Allocate from a bound thread. | ## Output 1. The NUMA topology and distance matrix. 2. A `numactl` binding command. 3. libnuma API calls for the use case. 4. A per-node data structure and affinity pattern. 5. A remote access measurement and a decision tree.