Walkeros Understanding Flow

Understanding walkerOS Flow

Overview

walkerOS follows a Source → Collector → Destination(s) architecture for composable, modular event processing.

Core principle: Separation of concerns. Each component has one job. Components are composable and replaceable.

The Flow Pattern

Sources → [Pre-Transformers] → Collector → [Post-Transformers] → Destinations
(Capture)  (source.next)    (Processing) (dest.before)       (Delivery)

- Browser DOM              - Validation   - Validation        - Google Analytics
- DataLayer                - Enrichment   - Enrichment        - Meta Pixel
- Server HTTP              - Redaction    - Consent check     - Custom API
- Cloud Functions                         - Routing           - Data Warehouse

Key Concepts

Composability

A Flow combines components. You can:

• Use multiple sources feeding one collector
• Route events to multiple destinations
• Swap components without changing others

The Flow Type

See packages/core/src/types/flow.ts for the canonical interface.

// Conceptual structure (see source for full type)
interface Flow {
  sources?: Record<string, Source>;
  transformers?: Record<string, Transformer>;
  destinations?: Record<string, Destination>;
  collector?: Collector.InitConfig;
}

Universal Push Interface

All components communicate via push functions:

The elb() function is an alias for collector.push - used for component wiring.

startFlow Helper

See packages/collector/src/flow.ts for the startFlow function.

import { startFlow } from '@walkeros/collector';

const { collector, elb } = await startFlow({
  sources: {
    /* ... */
  },
  transformers: {
    /* ... */
  },
  destinations: {
    /* ... */
  },
});

Separation of Concerns

Transformer Chains

Transformers run at two points in the pipeline, configured via next and before:

Pre-Collector Chain

Runs after source captures event, before collector processing:

Bundled mode (flow.json):

{
  "sources": {
    "browser": {
      "package": "@walkeros/web-source-browser",
      "next": "validate"
    }
  },
  "transformers": {
    "validate": {
      "package": "@walkeros/transformer-validator",
      "next": "enrich"
    },
    "enrich": {
      "package": "@walkeros/transformer-enricher"
    }
  }
}

Integrated mode (TypeScript):

sources: {
  browser: {
    code: sourceBrowser,
    next: 'validate'
  }
},
transformers: {
  validate: {
    code: transformerValidator,
    config: { next: 'enrich' }
  },
  enrich: {
    code: transformerEnrich
  }
}

Note: In flow.json, next is at the reference level. The CLI bundler automatically moves it into config.next for runtime - you don't need to handle this yourself.

Post-Collector Chain

Runs after collector enrichment, before destination receives event:

Bundled mode (flow.json):

{
  "destinations": {
    "gtag": {
      "package": "@walkeros/web-destination-gtag",
      "before": "redact"
    }
  },
  "transformers": {
    "redact": {
      "package": "@walkeros/transformer-redact"
    }
  }
}

Integrated mode (TypeScript):

destinations: {
  gtag: {
    code: destinationGtag,
    before: 'redact'
  }
},
transformers: {
  redact: {
    code: transformerRedact
  }
}

Chain Resolution

• source.next → starts pre-collector chain
• transformer.next (flow.json) or transformer.config.next (runtime) → links transformers
• destination.before → starts post-collector chain per destination

Related Skills

• walkeros-understanding-events - Event model
• walkeros-understanding-sources - Source interface
• walkeros-understanding-destinations - Destination interface

Package READMEs:

• packages/collector/README.md - Collector details

Source Files:

• packages/collector/src/flow.ts - startFlow implementation

Documentation:

• Website: Flow - Flow concept
• Website: Collector - Collector docs