Ruvector Attention Unified Wasm

@ruvector/attention-unified-wasm

Unified WebAssembly package providing 18+ attention mechanism implementations under a single API. Includes standard (Multi-Head, Flash), structural (DAG, Graph), state-space (Mamba SSM), sparse, linear, and cross-attention variants.

Quick Reference

Installation

npx @ruvector/attention-unified-wasm@latest

Node.js Usage

import init, {
  UnifiedAttention,
  DAGAttention,
  GraphAttention,
  MambaSSM,
} from '@ruvector/attention-unified-wasm';

await init();

const seqLen = 128;
const dim = 256;
const q = new Float32Array(seqLen * dim);
const k = new Float32Array(seqLen * dim);
const v = new Float32Array(seqLen * dim);

// Unified API: switch attention type by name
const flash = UnifiedAttention.forward(q, k, v, {
  type: 'flash',
  seqLen, dim,
  causal: true,
});

const sparse = UnifiedAttention.forward(q, k, v, {
  type: 'sparse',
  seqLen, dim,
  sparsityPattern: 'strided',
  blockSize: 16,
});

// DAG Attention: respect directed acyclic graph structure
const adjacency = new Uint8Array(seqLen * seqLen); // DAG adjacency matrix
const dagOut = DAGAttention.forward(q, k, v, {
  seqLen, dim,
  adjacency,
  topologicalSort: true,
});

// Graph Attention (GAT-style)
const nodeFeatures = new Float32Array(100 * 64); // 100 nodes, 64 features
const edges = new Uint32Array([0, 1, 1, 2, 2, 3]); // Edge list (pairs)
const graphOut = GraphAttention.forward(nodeFeatures, {
  numNodes: 100,
  featureDim: 64,
  edges,
  numHeads: 4,
  edgeDropout: 0.1,
});

// Mamba SSM: selective state-space model
const mamba = new MambaSSM({
  modelDim: 256,
  stateDim: 16,
  expandFactor: 2,
});

const output = mamba.forward(new Float32Array(seqLen * 256), { seqLen });

Browser Usage

<script type="module">
  import init, { UnifiedAttention } from '@ruvector/attention-unified-wasm';
  await init();

  const q = new Float32Array(64 * 128);
  const k = new Float32Array(64 * 128);
  const v = new Float32Array(64 * 128);
  const out = UnifiedAttention.forward(q, k, v, { type: 'flash', seqLen: 64, dim: 128 });
</script>

Key API

UnifiedAttention

Dispatch to any attention mechanism by type name.

UnifiedAttention.forward(
  q: Float32Array, k: Float32Array, v: Float32Array,
  config: UnifiedConfig
): Float32Array

UnifiedConfig:

Supported types:

DAGAttention

Attention respecting directed acyclic graph topology.

DAGAttention.forward(
  q: Float32Array, k: Float32Array, v: Float32Array,
  config: DAGConfig
): Float32Array

DAGConfig:

GraphAttention

Graph Attention Network (GAT) forward pass.

GraphAttention.forward(
  nodeFeatures: Float32Array,
  config: GATConfig
): Float32Array

GATConfig:

MambaSSM

Selective State-Space Model (Mamba architecture).

const mamba = new MambaSSM(config: MambaConfig);

MambaConfig:

mamba.forward(input: Float32Array, config: { seqLen: number }): Float32Array
mamba.stepDecoding(token: Float32Array): Float32Array  // Single-step for generation
mamba.resetState(): void
mamba.free(): void

UnifiedAttention.list()

List all supported attention types.

UnifiedAttention.list(): string[]

References

• API Reference
• npm