Ruvector Attention Wasm Pkg

ruvector-attention-wasm

High-performance WebAssembly attention mechanisms optimized for transformer models and LLMs. Provides MultiHead, Flash, and Hyperbolic attention implementations that run in browsers, Node.js, and edge runtimes.

Quick Reference

Installation

Hub install (recommended): npx agentdb@latest includes this package. Standalone: npx ruvector-attention-wasm@latest

Node.js Usage

import init, {
  MultiHeadAttention,
  FlashAttention,
  HyperbolicAttention,
} from 'ruvector-attention-wasm';

await init();

// Multi-Head Attention
const mha = new MultiHeadAttention({
  numHeads: 8,
  headDim: 64,
  dropout: 0.1,
});

const seqLen = 128;
const dim = 512;
const q = new Float32Array(seqLen * dim);  // Query
const k = new Float32Array(seqLen * dim);  // Key
const v = new Float32Array(seqLen * dim);  // Value

const output = mha.forward(q, k, v, { seqLen, dim });
console.log(`Output shape: ${seqLen} x ${dim}`);

// Flash Attention (memory-efficient, O(N) memory)
const flashOutput = FlashAttention.forward(q, k, v, {
  seqLen,
  dim,
  blockSize: 64,
  causal: true,
});

// Hyperbolic Attention (for hierarchical data)
const hyperOutput = HyperbolicAttention.forward(q, k, v, {
  seqLen,
  dim,
  curvature: -1.0,
});

Browser Usage

<script type="module">
  import init, { FlashAttention } from 'ruvector-attention-wasm';
  await init();

  const q = new Float32Array(64 * 256);
  const k = new Float32Array(64 * 256);
  const v = new Float32Array(64 * 256);
  const out = FlashAttention.forward(q, k, v, { seqLen: 64, dim: 256, causal: true });
</script>

Key API

MultiHeadAttention

Standard scaled dot-product attention with multiple heads.

const mha = new MultiHeadAttention(config: MHAConfig);
const output = mha.forward(q: Float32Array, k: Float32Array, v: Float32Array, shape: ShapeInfo): Float32Array;

MHAConfig:

ShapeInfo:

FlashAttention

Memory-efficient attention with tiled computation. Uses O(N) memory instead of O(N^2).

FlashAttention.forward(
  q: Float32Array, k: Float32Array, v: Float32Array,
  config: FlashConfig
): Float32Array

FlashConfig:

HyperbolicAttention

Attention in hyperbolic space for hierarchical and tree-structured data.

HyperbolicAttention.forward(
  q: Float32Array, k: Float32Array, v: Float32Array,
  config: HyperbolicConfig
): Float32Array

HyperbolicConfig:

Utility Functions

import { softmax, scaledDotProduct, attentionMask } from 'ruvector-attention-wasm';

// Softmax over a flat array
const probs = softmax(logits: Float32Array, dim: number): Float32Array;

// Scaled dot-product attention (single head)
const attn = scaledDotProduct(q: Float32Array, k: Float32Array, v: Float32Array, dim: number): Float32Array;

// Create causal attention mask
const mask = attentionMask(seqLen: number, causal: boolean): Float32Array;

References

• API Reference
• npm