Ruvector Burst Scaling

@ruvector/burst-scaling

Adaptive burst scaling system for RuVector deployments. Automatically handles 10-50x traffic spikes through request queuing, worker pool scaling, circuit breakers, and backpressure mechanisms.

Quick Reference

Installation

npx @ruvector/burst-scaling@latest

Quick Start

import {
  BurstScaler,
  CircuitBreaker,
  BackpressureQueue,
} from '@ruvector/burst-scaling';

// Wrap any async handler with burst scaling
const scaler = new BurstScaler({
  baseWorkers: 4,
  maxWorkers: 64,
  scaleUpThreshold: 0.8,    // Scale at 80% utilization
  scaleDownThreshold: 0.2,  // Scale down at 20%
  maxQueueSize: 10_000,
  targetLatencyMs: 50,
});

// Wrap a search handler
const scaledSearch = scaler.wrap(async (query: Float32Array) => {
  return index.search(query, 10);
});

// Handle requests - automatically scales under load
const results = await scaledSearch(queryVector);

// Monitor scaling behavior
const metrics = scaler.metrics();
console.log(`Workers: ${metrics.activeWorkers}/${metrics.maxWorkers}`);
console.log(`Queue depth: ${metrics.queueDepth}`);
console.log(`P99 latency: ${metrics.p99LatencyMs}ms`);

Core API

BurstScaler

Main auto-scaling controller.

const scaler = new BurstScaler(config: ScalerConfig);

ScalerConfig:

scaler.wrap(handler)

Wrap an async function with automatic scaling.

scaler.wrap<T, R>(handler: (input: T) => Promise<R>): (input: T) => Promise<R>

scaler.wrapBatch(handler)

Wrap a batch handler that processes arrays.

scaler.wrapBatch<T, R>(handler: (inputs: T[]) => Promise<R[]>): (input: T) => Promise<R>

Automatically batches individual requests for efficiency.

scaler.metrics()

Get current scaling metrics.

scaler.metrics(): ScalerMetrics

ScalerMetrics:

scaler.setLimits(config)

Dynamically update scaling limits.

scaler.setLimits(config: Partial<ScalerConfig>): void

scaler.pause() / scaler.resume()

scaler.pause(): void   // Stop accepting new requests
scaler.resume(): void  // Resume accepting requests

scaler.shutdown()

Graceful shutdown: drain queue, stop workers.

await scaler.shutdown(): Promise<void>

CircuitBreaker

Prevent cascading failures under sustained load.

const breaker = new CircuitBreaker(config: BreakerConfig);

BreakerConfig:

const protected = breaker.wrap(riskyFn);
const result = await protected(input);

breaker.state: 'closed' | 'open' | 'half-open'
breaker.reset(): void
breaker.on('open', () => { /* alert */ })
breaker.on('close', () => { /* recovered */ })

BackpressureQueue

Request queue with configurable backpressure strategies.

const queue = new BackpressureQueue(config: QueueConfig);

QueueConfig:

queue.enqueue(request): Promise<void>
queue.size(): number
queue.isFull(): boolean
queue.drain(): Promise<void>

Common Patterns

Auto-Scaling Vector Search

const scaler = new BurstScaler({ baseWorkers: 4, maxWorkers: 32 });
const search = scaler.wrap(async (q: Float32Array) => index.search(q, 10));

// Handle burst traffic
app.post('/search', async (req, res) => {
  const results = await search(req.body.query);
  res.json(results);
});

Circuit Breaker with Fallback

const breaker = new CircuitBreaker({ failureThreshold: 3 });
const protectedSearch = breaker.wrap(remoteSearch);

async function searchWithFallback(query) {
  try {
    return await protectedSearch(query);
  } catch (e) {
    return localCache.search(query); // Fallback
  }
}

Events

scaler.on('scale:up', (from, to) => console.log(`Scaled ${from} -> ${to} workers`));
scaler.on('scale:down', (from, to) => console.log(`Scaled ${from} -> ${to} workers`));
scaler.on('queue:full', () => console.log('Queue at capacity'));
scaler.on('request:timeout', (req) => console.log('Request timed out'));

References

• API Reference
• npm