KunanonJ Authoring Signals scouts
A scout is a scheduled agent that wakes on its own interval, looks at one PostHog project, decides what's genuinely worth surfacing, and emits it as a finding into the Signals inbox — or closes out empty, which is a real outcome. PostHog ships a fleet of canonical scouts (a cross-product generalist plus per-surface specialists). This skill helps you and your agent adapt those canonical scouts to a specific project, or author new scouts from scratch for a use case the fleet doesn't cover.
A scout is just an LLMSkill whose name starts with signals-scout-. The harness
discovers scouts by globbing signals-scout-* over the project's skills, loads the body
verbatim as the agent's system prompt, and progressively reads any bundled reference
files on demand. The signals-scout- name prefix is load-bearing: a skill named
anything else will never run as a scout.
The job before the writing
Don't write a scout in the abstract. Ground it in the target project first — a scout is only as good as its fit to the data it watches.
- Read the project.
posthog:signals-scout-project-profile-getreturns the deterministic snapshot the scout itself cold-starts from: products in use, top events with reach/burst metrics, integrations, existing inbox counts. If the scout watches a specific event, confirm it exists and check its shape withposthog:read-data-schema. A scout for an event the project doesn't capture is dead on arrival. - See what already runs.
posthog:signals-scout-config-listlists every existing scout on the project with its schedule,enabled, andemitposture. Don't duplicate a surface a canonical scout already covers — adapt that one instead. - Read the closest canonical scout. It's your template and your reference shape. Pull
it with
posthog:llma-skill-get {"skill_name": "signals-scout-<x>"}(per-team rows) or read it from the repo atproducts/signals/skills/signals-scout-*/. The generalist (signals-scout-general) is the broad template; pick a specialist (-error-tracking,-ai-observability,-logs,-revenue-analytics,-surveys,-csp-violations,-observability-gaps) if your scope is domain-tight. - Skim the inbox.
posthog:inbox-reports-listshows what findings are actually landing — calibrate so your scout adds signal, not noise.
Choose the path
There are two independent decisions: what you're building, and where it lives.
What
| Situation | Approach |
|---|---|
| A canonical scout is close but too broad / too noisy / missing a disqualifier for this project | Adapt it — narrow the scope, add disqualifiers, retune thresholds. |
| You want a surface no canonical scout covers (a custom event, a product-specific funnel) | New scout from scratch — copy the closest canonical scout as scaffolding, replace the domain discriminator + explore patterns. |
| You only want to change when / whether a scout runs | No authoring — just tune the config (see Run posture). |
Where
| Path | Mechanism | Use when |
|---|---|---|
| Per-team (the common user path) | Create/edit a signals-scout-* LLMSkill row in the project's skills store via posthog:llma-skill-create / -update / -file-create. |
Customizing for one project. The harness globs the row in on the next tick; canonical sync leaves your edited ("diverged") row alone. |
| Canonical (PostHog contributors) | Edit disk under products/signals/skills/signals-scout-*/, lint/build, open a PR. |
Improving a scout for every enrolled project. lazy_seed mirrors it onto all enrolled teams on the next tick. |
Adapting-in-place tradeoff: editing a canonical scout's row for your team marks it
diverged — you stop receiving upstream improvements to that scout. If you only need an
additional behavior, prefer authoring a new, differently-named scout
(signals-scout-<your-scope>) and leaving the canonical one intact.
See references/lifecycle-and-testing.md for the
exact skills-store calls, the build/lint commands, and how seeding works.
Write the scout
Follow references/scout-anatomy.md — it has the frontmatter
schema, the canonical body structure (quick close-out → orient → domain discriminator →
explore patterns → save-memory → decide → disqualifiers → close-out), the lean-body rule,
and copy-ready skeleton templates for both a specialist and the generalist.
Two craft references the whole fleet reasons in terms of — a good scout's Decide and memory sections are built on them, so read them before writing those sections:
references/emit-contract.md— whatemit-signaltakes, the weight vs. confidence rubrics, severity, dedupe keys,finding_id, the description prose contract, and a worked example. This is how your scout decides what clears the bar and how to write the finding.references/dedupe-and-memory.md— the four-states classifier (net-new / material-update / already-covered / addressed-or-noise), the scratchpad key-prefix vocabulary, and the cross-project noise patterns. This is how your scout avoids re-emitting and learns across runs.
The single most important design decision in any scout is its signal-vs-noise
discriminator — the cheap profile-shape read that separates "worth investigating" from
"baseline". For error tracking it's the count vs distinct_users ratio; for CSP it's
reach over raw count. Your new scout needs its own. Name it explicitly near the top of the
body so every run anchors on it.
Run posture (config)
A scout's schedule and emit behavior live on its SignalScoutConfig, separate from the
skill body. Tune with posthog:signals-scout-config-update (find the id via
-config-list):
run_interval_minutes— 10 to 43200. Default 60 (hourly). Slow a chatty or expensive scout by raising this.enabled—falsepauses the scout entirely (coordinator skips it).emit—false= dry-run: the scout runs and logs its reasoning but writes nothing to the inbox. New and freshly-edited scouts should run dry-run first.
Test loop
You can't force a synchronous run as a user — scouts fire on their schedule. The feedback loop is dry-run + inspect:
- Ship the scout with
emit=falseand a shortrun_interval_minutesso it fires soon. - After a tick, read what it did:
posthog:signals-scout-runs-list(run summaries),-runs-retrieve(full reasoning for one run), and-scratchpad-search(the durable memory it wrote). In dry-run, runs show what it would have emitted. - Refine the body — tighten the discriminator, add disqualifiers for whatever it false-positived on, fix the emit calibration.
- When the dry-run findings look right, flip
emit=true. Restore the interval to something sustainable (hourly+).
Repo contributors get a faster loop — hogli sync:skill and the harness's local run path;
see references/lifecycle-and-testing.md.
To read what your scouts are doing rather than change them — surveying the fleet, inspecting
individual runs, the scratchpad memory, and assessing performance — use the read-only companion
skill exploring-signals-scouts. Keep the two in sync when
the scout config / run / scratchpad surfaces change.
Quality bar for a v1 scout
- A named, cheap signal-vs-noise discriminator anchored near the top.
- A quick close-out so a quiet run is cheap (don't pay for deep exploration when the watched surface is at baseline or absent).
- 2–4 concrete explore patterns with the actual queries/tools to run — starting points, not a rigid checklist.
- Disqualifiers listing this project's known noise (single-user quirks, dev-env bursts, allowlisted entities).
- A Decide section calibrated against the emit contract (confidence ≥ 0.65 to emit; below that, write memory).
- Save-memory guidance using the scratchpad prefixes so the scout gets smarter each run.
- A lean body (push depth into
references/) — every line is a recurring token cost on every run.