Simd Intrinsics

SIMD intrinsics

Let the compiler vectorize first; write intrinsics where it cannot. Every intrinsic code path needs a runtime feature check and a scalar fallback, because the binary runs on CPUs the build machine never saw.

Contract

Inputs

• The hot loop or kernel: required.
• The compiler and flags: required. Vectorization reports differ between GCC and Clang.
• The deployment CPU range: required before choosing the ISA level.

Procedure

1. Check whether the loop vectorizes already. Clang prints remarks with -Rpass, -Rpass-missed, and -Rpass-analysis; GCC prints with -fopt-info-vec, -fopt-info-vec-missed, and -fopt-info-vec-optimized. Done when: the loop has a verdict with the reason.

clang -O2 -march=native -Rpass-missed=loop-vectorize src.c
gcc -O2 -march=native -fopt-info-vec-missed src.c

2. Fix the loop before reaching for intrinsics. The usual blockers and their fixes:

void addf(float *__restrict dst,
          const float *__restrict a, const float *__restrict b, size_t n) {
    // restrict promises no aliasing, so the compiler may reorder loads
}

3. Detect CPU features at runtime before any intrinsic path runs. Done when: dispatch selects the widest ISA the running CPU supports.

#include <stdbool.h>

bool have_avx2(void) {
    return __builtin_cpu_supports("avx2");    // GCC and Clang
}

For finer control, use __cpuid with __builtin_cpu_supports covering the common cases, and check xgetbv state through the compiler builtin for OS AVX support rather than reading registers by hand.

4. Write the x86 paths. SSE2 moves four floats per __m128; AVX2 moves eight per __m256. Unaligned loads on modern x86 cost little; use them unless profiling proves otherwise. Done when: each path compiles under its own feature guard and the tail stays correct.

#include <immintrin.h>

#ifdef __AVX2__
void sum8_avx2(float *dst, const float *a, const float *b, int n) {
    int i = 0;
    for (; i + 8 <= n; i += 8) {
        __m256 va = _mm256_loadu_ps(a + i);      // unaligned load, 8 floats
        __m256 vb = _mm256_loadu_ps(b + i);
        __m256 vc = _mm256_add_ps(va, vb);
        _mm256_storeu_ps(dst + i, vc);
    }
    for (; i < n; i++)                           // scalar tail
        dst[i] = a[i] + b[i];
}
#endif

FMA combines multiply and add and needs its own feature; guard it:

#if defined(__FMA__)
    __m256 vsum = _mm256_fmadd_ps(va, vb, vacc);  // vacc + va*vb
#endif

Compile the AVX2 translation unit with -mavx2 -mfma, or mark the single function:

__attribute__((target("avx2,fma")))
void kernel_avx2(float *dst, const float *a, const float *b, int n) { }

5. Write the NEON path for ARM. NEON is baseline on AArch64, so no runtime check is needed there. Done when: the NEON path builds for aarch64 and the 128-bit vector shape matches.

#include <arm_neon.h>

void sum4_neon(float *dst, const float *a, const float *b, int n) {
    for (int i = 0; i + 4 <= n; i += 4) {
        float32x4_t va = vld1q_f32(a + i);
        float32x4_t vb = vld1q_f32(b + i);
        vst1q_f32(dst + i, vaddq_f32(va, vb));
    }
}

6. Decide between intrinsics and compiler builtins. Reach for intrinsics when the code needs shuffles, gathers, or horizontal reduction that auto-vectorization will not find, or when the report from step 1 names a pattern the compiler refuses. Prefer compiler auto-vectorization plus builtins like __builtin_popcount for element-wise arithmetic. Done when: the choice is written next to the kernel.

7. Mind alignment where it matters. Heap buffers can be aligned with aligned_alloc(32, ...) for AVX2, and posix_memalign covers POSIX builds. Unaligned loads are correct everywhere and nearly free on recent x86, so pay for alignment only with a measurement. Done when: alignment claims carry the measurement that justifies them.

8. Measure on the target machine. Compare against the scalar baseline with the real data size and record the counters or timing. Done when: the speedup number exists and names the machine and build flags.

Failure and recovery

Output

The guarded vector kernel with its scalar fallback, the dispatch check, and the measured speedup with the build flags and machine named. The intrinsic lookup tables, type maps, and feature guard reference are in references/intel-intrinsics-guide.md.