Ruvector Ruvllm Wasm

@ruvector/ruvllm-wasm

WebAssembly bindings for browser-native LLM inference, enabling text generation, embeddings, and streaming completions directly in the browser with WebGPU acceleration and quantized model support.

Quick Reference

Installation

Install: npx @ruvector/ruvllm-wasm@latest See Installation Guide for the full ecosystem.

Key API

WasmLLM

Main WASM-accelerated LLM inference engine.

Node.js:

import { WasmLLM } from '@ruvector/ruvllm-wasm';

const llm = new WasmLLM({
  model: 'tinyllama-1.1b-q4',
  maxTokens: 256,
});

Browser:

import init, { WasmLLM } from '@ruvector/ruvllm-wasm';

await init(); // Initialize WASM module

const llm = new WasmLLM({
  model: 'tinyllama-1.1b-q4',
  maxTokens: 256,
  webgpu: true,
});

Constructor Options:

Methods:

WasmTokenizer

Standalone WASM tokenizer.

import init, { WasmTokenizer } from '@ruvector/ruvllm-wasm';
await init();

const tokenizer = new WasmTokenizer({ model: 'tinyllama-1.1b-q4' });
const tokens = tokenizer.encode('Hello, world!');
const text = tokenizer.decode(tokens);

Methods:

WasmEmbedder

Dedicated embedding generator.

import init, { WasmEmbedder } from '@ruvector/ruvllm-wasm';
await init();

const embedder = new WasmEmbedder({ model: 'all-minilm-l6-q4' });
const embedding = await embedder.embed('search query');
const similarity = embedder.cosineSimilarity(emb1, emb2);

Constructor Options:

Methods:

Common Patterns

Browser Chat Interface

import init, { WasmLLM } from '@ruvector/ruvllm-wasm';

await init();
const llm = new WasmLLM({ model: 'tinyllama-1.1b-q4', webgpu: true });

const outputEl = document.getElementById('output');
for await (const token of llm.stream(userInput)) {
  outputEl.textContent += token;
}

Offline-Capable PWA

import init, { WasmLLM } from '@ruvector/ruvllm-wasm';

await init();

// Cache model in IndexedDB on first load
const cache = await caches.open('llm-models');
let modelData = await cache.match('/models/tinyllama-q4.bin');
if (!modelData) {
  modelData = await fetch('/models/tinyllama-q4.bin');
  await cache.put('/models/tinyllama-q4.bin', modelData.clone());
}

const llm = new WasmLLM({ model: 'tinyllama-1.1b-q4' });
await llm.loadModel(await modelData.arrayBuffer());

Semantic Search in Browser

import init, { WasmEmbedder } from '@ruvector/ruvllm-wasm';

await init();
const embedder = new WasmEmbedder({ model: 'all-minilm-l6-q4' });

const docEmbeddings = await embedder.embedBatch(documents);
const queryEmb = await embedder.embed(searchQuery);

const scores = docEmbeddings.map((emb, i) => ({
  index: i,
  score: embedder.cosineSimilarity(queryEmb, emb),
}));
scores.sort((a, b) => b.score - a.score);

RAN DDD Context

Bounded Context: Learning

References

• API reference: See references/commands.md
• Full README
• npm