Output Streams

Output Streams

LimaCharlie routes data through four distinct output streams. Each has a different schema — they are NOT interchangeable. Understanding these structures is essential for configuring outputs, building parsers in external systems, and filtering data.

Stream Overview

Event Stream

Canonical two-level structure. Every event has routing (metadata) and event (payload):

{
  "routing": {
    "oid": "8cbe27f4-...",
    "sid": "bb4b30af-...",
    "event_type": "NEW_PROCESS",
    "event_time": 1656959942437,
    "event_id": "8cec565d-...",
    "hostname": "workstation-01",
    "iid": "7d23bee6-...",
    "did": "b97e9d00-...",
    "ext_ip": "203.0.113.45",
    "int_ip": "10.0.1.25",
    "plat": 268435456,
    "arch": 2,
    "this": "a443f9c48bef700740ef27e062c333c6",
    "parent": "42217cb0326ca254999554a862c3298e",
    "tags": ["production"]
  },
  "event": {
    "FILE_PATH": "C:\\Windows\\System32\\cmd.exe",
    "COMMAND_LINE": "cmd.exe /c whoami",
    "PROCESS_ID": 4812,
    "USER_NAME": "Administrator",
    "PARENT": {
      "FILE_PATH": "C:\\Windows\\explorer.exe",
      "PROCESS_ID": 2156
    }
  }
}

Key event types: NEW_PROCESS, DNS_REQUEST, NETWORK_CONNECTIONS, FILE_MODIFIED, WEL (Windows Event Logs), MODULE_LOAD, REGISTRY_WRITE, SENSITIVE_PROCESS_ACCESS.

Detection Stream

Alerts from D&R rules. Different structure from events — flat object with detection-specific fields:

{
  "cat": "Suspicious PowerShell Execution",
  "source": "dr-general",
  "routing": { "..." },
  "detect": {
    "FILE_PATH": "C:\\...\\powershell.exe",
    "COMMAND_LINE": "powershell.exe -enc SGVsbG8="
  },
  "detect_id": "f1e2d3c4-...",
  "priority": 7,
  "detect_mtd": { "rule_name": "detect-encoded-powershell" },
  "detect_data": {
    "suspicious_process": "powershell.exe",
    "encoded_command": "SGVsbG8="
  },
  "source_rule": "detect-encoded-powershell",
  "rule_tags": ["windows", "powershell"],
  "gen_time": 1656959942500
}

Detection Fields

detect_data — Structured IOCs

The detect_data field contains pre-parsed IOCs extracted by the rule. This is the primary integration point for:

• Automated enrichment (lookup IPs, domains, hashes)
• SOAR playbook inputs
• Case management systems
• Threat intelligence platforms

Audit Stream

Platform management events. Completely different structure — flat object, does NOT use the routing/event split:

{
  "oid": "8cbe27f4-...",
  "ts": "2024-06-05T14:23:18Z",
  "etype": "config_change",
  "msg": "D&R rule created",
  "origin": "api",
  "time": 1656959942,
  "ident": "user@company.com",
  "entity": {
    "type": "dr_rule",
    "name": "detect-encoded-powershell",
    "hive": "dr-general"
  },
  "mtd": {
    "action": "create",
    "source_ip": "203.0.113.10"
  }
}

Audit Fields

Deployment Stream

Sensor lifecycle events. Uses routing/event structure like events but with deployment-specific event types:

Use deployment events to detect unexpected sensor removals (potential evasion), track sensor versions, and monitor fleet coverage.

Output Configuration

CLI Commands

# List outputs
limacharlie output list --oid <oid> --output yaml

# Create output
cat > /tmp/output.yaml << 'EOF'
name: my-syslog
module: syslog
for: detect
dest_host: siem.example.com:514
EOF
limacharlie output create --input-file /tmp/output.yaml --oid <oid> --output yaml

# Delete output
limacharlie output delete --name <name> --confirm --oid <oid>

Stream Filtering

Filter parameters use newline-separated string format, NOT YAML arrays.

# Event type filtering (whitelist)
event_white_list: |
  NEW_PROCESS
  DNS_REQUEST

# Event type filtering (blacklist)
event_black_list: |
  CONNECTED
  DNS_REQUEST

# Detection category filtering
cat_white_list: |
  high-priority
  critical

# Tag-based filtering (single tag)
tag: production

# Tag blacklist
tag_black_list: |
  dev
  test

# Rule tag filtering (detection stream only)
rule_tag_white_list: |
  compliance
  ransomware

Tailored Output Stream

The output D&R response action forwards a matched event to a specific output in the tailored stream. This enables highly granular routing — e.g., send only specific detection types to a specific webhook:

- action: output
  name: my-webhook-output

Gotchas Summary