Go Collector Design
Use this skill to design or review what a collector promises, who owns its state, what an operator decides, what a
sample means, and what proves it. Resolve applicable implementation design questions before code. Mechanics live in
.agents/skills/collectors-go-framework-v2/SKILL.md and
src/go/plugin/go.d/docs/how-to-write-a-collector.md; artifact delivery lives in
.agents/skills/integrations-lifecycle/. Do not restate those here.
Every rule below is written as: When it applies, Do / Don't, what counts as Evidence, and its Boundary
(the legitimate exception). A checked box or an approval phrase is never evidence.
Review And Implementation
Apply AGENTS.md#skill-selection. During review, use the applicable items as questions about the changed contract,
source and existing design/validation evidence. Report concrete missing or inconsistent evidence; do not create a SOW,
design note, truth-table artifact or mutation test merely because this authoring workflow describes it. Required
acceptance evidence still matters. Loading a reference does not authorize its UI changes, remote operations or setup.
During authorized implementation, record the applicable design before code and retain the project's approval gates.
A pure form presentation edit uses the schema reference without a collector design note; an option/default/meaning
change remains design work even when implemented through the schema. An ordinary transient cache does not by itself
require durable-state machinery; ownership, lifecycle and cost changes still need their applicable design review.
When This Skill Applies
| Task |
Load |
Design note depth |
| New go.d collector |
this skill, then the V2 skill and the how-to guide |
full note; one line per item for a small read-only collector |
| New public config option, mode, or default change |
operator-surface.md (the option's decision record) |
the affected item only |
Form presentation in config_schema.json, with option/default semantics unchanged |
config-schema.md |
none; a changed operator contract uses the relevant design row |
| New or changed metric meaning, new entity axis, vnodes |
the Metric Semantics and Identity items |
the affected item only |
| Collector that writes or deletes remotely, or persists durable local state |
this skill plus applicable mutating-collectors.md sections |
full note plus applicable state/mutation items |
| Reviewing any of the above |
affected items as review questions and existing evidence |
no new implementation artifact |
| Contract-preserving migration/fix or integration prose only |
migration/V2 or metadata/integration guidance as applicable |
none; schema forms remain routed above |
The Collector Design Note
When: authorized design or implementation of a new collector or an affected public, ownership or lifecycle
contract. Do: fill the applicable items below as a Collector design: block under
"Affected contracts and surfaces" in the SOW's Pre-Implementation Gate, before implementation. Don't: create
separate documents, or answer items the collector does not have; write "none" with the reason instead. Evidence:
each item cites its source (provider doc, existing code, framework contract, user decision). Boundary: a small
collector that only reads its source answers applicable items in one line each. Pure form edits and review-only
requests follow the exceptions above. Remote mutation or durable local state selects the applicable sections of
mutating-collectors.md; it does not make every item apply to every collector.
- Product boundary. State the operational question the collector answers, the supported providers, versions,
and configurations, the explicit non-goals, and whether each measurement is client-observed or a backend
guarantee. Separate three things that get confused: a repair (the contemporary contract was violated), a
discoverability fix (the contract was hidden), and an expansion (a new capability, which needs the user's
approval). An explicitly excluded capability is not a defect.
- Provider contract. For every operation the design depends on, name the permissions, consistency assumptions,
retries, and error meanings, with a link to the applicable provider/version documentation. When more than one
provider or mode is involved, fill a capability matrix: one row per operation, one column per provider, cells say
supported / semantics / evidence. An S3-compatible request API does not imply interchangeable replication,
versioning, or deletion semantics. Similar charts may share observations while provider operations differ; decide
sharing from the matrix, not from vendor count or API naming.
- Architecture and ownership. Name what owns config, client transport, normalization, durable state, and
presentation; which existing helpers fit (
src/go/plugin/go.d/docs/helper-packages.md); what needs a boundary and
what stays direct code. Any coupling across jobs or owners, any durable state, any scheduler or queue goes through
the Architecture Gate below first.
- Identity and lifecycle. Name what survives a cycle, a restart, and a reload; which identity is stable and which
is display metadata; which concurrent owners actually exist (the same job's old and new runtime is one case,
different jobs another). Trace one successful cycle and one cycle with unfinished cleanup before designing the
engine. If housekeeping would stop measurement, either state the real dependency or separate the two state
dimensions; do not shorten a safety interval to hide the stall.
- Operator surface. One decision record row per proposed option, the mode form as a user task, and the
consumer traces for defaults and null:
operator-surface.md. List the implementation details you intentionally do
not expose.
- Metric semantics. One measurement truth table row per new observation (below). Derive names, units, and help
text from the table, never the other way round.
- Evidence plan. What proves the real path (real construction, real transitions, the shipped adapter), which
fakes carry independent semantics, and what cannot be verified locally and is therefore stated as unverified. Test
rules live in the V2 skill's Tests section.
Architecture Gate
For configured vnode acquisition and named attachment, use
src/go/plugin/framework/vnodes/README.md#ownership and
src/go/plugin/framework/vnodes/README.md#collector-attachment as the existing ownership contract before proposing
collector-owned identity polling or shared connection settings.
When: a proposal makes one job depend on another job's state (scanning its journals, waiting for its cleanup,
sharing an operational lock, consulting a registry for permission to run), introduces durable state for otherwise
independent reads, adds a scheduler or queue, lets cleanup freeze measurement, or builds a generic engine around one
provider's quirks. Do: answer the five questions in the design note before code; if the answer is "unknown",
investigate or ask, never implement a placeholder for later review. Don't: repair the proposal by making the scan
faster, raising a file cap, adding retries, or writing tests that expect the coupling; those preserve the wrong
dependency. Evidence: concrete object keys, owner identities, and a named invariant, not "there could be races".
Boundary: a genuinely shared resource may require coordination; the gate is not a ban on locks, shared clients,
or framework infrastructure, and "this looks complex" is not a finding without a named dependency and consequence.
- Shared resource. What concrete object, namespace, limit, or external protocol is shared? Sharing a directory,
an SDK, or a provider type is not a collision.
- Necessity. Which supported execution fails with independent per-job ownership? Show the collision.
- Narrowest boundary. Why is per-owner identity and exclusion insufficient? Compare with the proposal using real
keys and owner identities.
- Failure propagation and cost. Can a stopped, corrupt, or unreachable job block a healthy one? State the cost
variables: work per job per call, per retained item, remote calls, state serialization, lock scope, growth with
jobs and backlog. Use source-derived bounds at design time; a shipped hot-path change still follows
src/go/AGENTS.md "Hot-Path And Benchmark Discipline". A cache that preserves the failure coupling is not a fix.
- Decision. Necessary coordination is exposed as an operational trade-off and gets the applicable design
approval. Unnecessary coordination is redesigned around the actual owner boundary. For durable ownership also
state the recovery consequences: same owner versus different owner, label change, credential rotation, location
change, rename. Isolation does not solve identity migration; say what a renamed job does and does not inherit.
Worked example, from the S3check original: the proposal scanned every job's ownership files under a global handoff
lock before publishing a probe. Q1: object keys were already namespaced per Agent and per job, so no key collided;
a shared directory is not a shared resource. Q2: no supported execution failed with per-job ownership; the only real
overlap was the same job's old and new runtime during reload. Q3: a per-owner lock and journal (owner = Agent
registry ID + job name) covered that overlap. Q4: an unrelated corrupt journal blocked a healthy job, scanning was
O(jobs²), and a 256-file cap turned growth into a hard failure. Decision: reject the coordination, redesign around
owner identity, record that a renamed job does not adopt old ownership. All of this was decidable from the proposal.
Measurement Truth Table
When: every new observation, and every change to what an existing one means. Do: fill one row per
observation, then map states to values, before writing metric names or charts.yaml. Don't: emit a value that
looks like a measurement for something not measured; a skipped operation has no duration, a failed attempt is not a lag
or success sample (its request duration may still be a valid measurement of the request), a waiting state is not a zero
(.agents/skills/collectors-authoring/collector-practices.md#14-gaps-are-data). Evidence: the table itself, plus a
test that drives each row's state through the real path and asserts emitted / omitted / retained. Boundary:
human-readable configuration does
not prohibit millisecond latency charts; the table decides units per chart.
| Column |
Meaning |
| Eligibility |
when the observation may be produced at all |
| Start event |
what starts the measurement (for S3check delete lag: the successful source DELETE, not the first attempt) |
| Stop event |
what ends it (observed destination absence; a timeout ends the attempt, a breach does not) |
| Scope |
per probe, per target, per job |
| Aggregation |
how repeated calls become one value or status before chart labels collapse them |
| Consequence |
what the operator should conclude from it |
Then map each of: measured, missing, skipped, waiting, retrying, failed operation, failed collection, backpressured,
terminal, to one of: emitted value, omitted (gap), retained last terminal value, state-set state. Distinguish a
measured unhealthy target from an inability to collect; make sure the framework commits the intended failure
observations; check every early return against the table. Comparisons follow the wording ("exceeding" is strict).
Lifecycle Entry Points
When: designing Init, Check, Collect, Cleanup, and any Run. Do: review every entry point, including
partial initialization, DynCfg test, autodetection, reload, and stop, not only Collect followed by a clean
shutdown. Decide, per entry point, what it may do, and record it; the V2 skill's Core Style owns the resulting rules
(Check detection-only, cleanup caller-cancelled or detached with a fixed budget independent of request and retry
settings; S3check chose five seconds). Don't: derive a shutdown budget from public tuning, or make orderly cleanup
the crash-recovery mechanism. Evidence: the trace per entry point in the design note; tests for cancellation and
partial init. Boundary: a read-only collector closing idle connections needs no journal or timeout machinery;
background contexts are not banned, unbounded ones are.
Simplification As Engineering
When: throughout, and at the final pass. Do: prefer direct ownership, small types, clear transitions, and
existing helpers over generic engines and defensive layers; name files by responsibility and check that the content
matches; split along operations or state boundaries, not line counts; share only real semantics and keep distinct
provider logic distinct; run the V2 skill's Pre-PR final sweep before review. Don't: reject necessary state or
boundaries to minimize the diff, or add pooling and caching to satisfy a slogan. Evidence: per-cycle cost stated
from the source (a global O(N²) scan matters; a bounded map in a network-bound check does not). Boundary: file
length is a signal, not a limit; splitting one function into arbitrarily named helpers is not architecture.
Reading Sequence
- Select affected items from the task table. Implementation records its applicable note; review checks existing
contracts and evidence under
./SKILL.md#review-and-implementation.
- Read
operator-surface.md for option/mode decisions and config-schema.md for form authoring or review.
Read mutating-collectors.md for remote mutation or durable local state, using its scoped routes.
- For implementation mechanics or their review, use
src/go/plugin/go.d/docs/how-to-write-a-collector.md and the V2
skill when that framework applies; contract-preserving migrations use the migration guide.
- When metadata or delivery is affected, use
.agents/skills/collectors-metadata-yaml/SKILL.md and
.agents/skills/integrations-lifecycle/consistency.md. Reading delivery guidance does not request regeneration.
1---2name: collectors-go-design-23description: Design or review go.d collector contracts, including options, metric meaning and identity, vnodes, Functions, ownership, remote writes and durable state. Also author or review DynCfg config_schema.json forms. Contract-preserving fixes and migrations use framework guidance; integration prose uses the metadata skill.4metadata:5 source: https://skillmd.com/skills/netdata/collectors-go-design-26---78# Go Collector Design910Use this skill to design or review what a collector promises, who owns its state, what an operator decides, what a11sample means, and what proves it. Resolve applicable implementation design questions before code. Mechanics live in12`.agents/skills/collectors-go-framework-v2/SKILL.md` and13`src/go/plugin/go.d/docs/how-to-write-a-collector.md`; artifact delivery lives in14`.agents/skills/integrations-lifecycle/`. Do not restate those here.1516Every rule below is written as: **When** it applies, **Do / Don't**, what counts as **Evidence**, and its **Boundary**17(the legitimate exception). A checked box or an approval phrase is never evidence.1819## Review And Implementation2021Apply `AGENTS.md#skill-selection`. During review, use the applicable items as questions about the changed contract,22source and existing design/validation evidence. Report concrete missing or inconsistent evidence; do not create a SOW,23design note, truth-table artifact or mutation test merely because this authoring workflow describes it. Required24acceptance evidence still matters. Loading a reference does not authorize its UI changes, remote operations or setup.2526During authorized implementation, record the applicable design before code and retain the project's approval gates.27A pure form presentation edit uses the schema reference without a collector design note; an option/default/meaning28change remains design work even when implemented through the schema. An ordinary transient cache does not by itself29require durable-state machinery; ownership, lifecycle and cost changes still need their applicable design review.3031## When This Skill Applies3233| Task | Load | Design note depth |34|---|---|---|35| New go.d collector | this skill, then the V2 skill and the how-to guide | full note; one line per item for a small read-only collector |36| New public config option, mode, or default change | `operator-surface.md` (the option's decision record) | the affected item only |37| Form presentation in `config_schema.json`, with option/default semantics unchanged | `config-schema.md` | none; a changed operator contract uses the relevant design row |38| New or changed metric meaning, new entity axis, vnodes | the Metric Semantics and Identity items | the affected item only |39| Collector that writes or deletes remotely, or persists durable local state | this skill plus applicable `mutating-collectors.md` sections | full note plus applicable state/mutation items |40| Reviewing any of the above | affected items as review questions and existing evidence | no new implementation artifact |41| Contract-preserving migration/fix or integration prose only | migration/V2 or metadata/integration guidance as applicable | none; schema forms remain routed above |4243## The Collector Design Note4445**When:** authorized design or implementation of a new collector or an affected public, ownership or lifecycle46contract. **Do:** fill the applicable items below as a `Collector design:` block under47"Affected contracts and surfaces" in the SOW's Pre-Implementation Gate, before implementation. **Don't:** create48separate documents, or answer items the collector does not have; write "none" with the reason instead. **Evidence:**49each item cites its source (provider doc, existing code, framework contract, user decision). **Boundary:** a small50collector that only reads its source answers applicable items in one line each. Pure form edits and review-only51requests follow the exceptions above. Remote mutation or durable local state selects the applicable sections of52`mutating-collectors.md`; it does not make every item apply to every collector.53541. **Product boundary.** State the operational question the collector answers, the supported providers, versions,55 and configurations, the explicit non-goals, and whether each measurement is client-observed or a backend56 guarantee. Separate three things that get confused: a repair (the contemporary contract was violated), a57 discoverability fix (the contract was hidden), and an expansion (a new capability, which needs the user's58 approval). An explicitly excluded capability is not a defect.592. **Provider contract.** For every operation the design depends on, name the permissions, consistency assumptions,60 retries, and error meanings, with a link to the applicable provider/version documentation. When more than one61 provider or mode is involved, fill a capability matrix: one row per operation, one column per provider, cells say62 supported / semantics / evidence. An S3-compatible request API does not imply interchangeable replication,63 versioning, or deletion semantics. Similar charts may share observations while provider operations differ; decide64 sharing from the matrix, not from vendor count or API naming.653. **Architecture and ownership.** Name what owns config, client transport, normalization, durable state, and66 presentation; which existing helpers fit (`src/go/plugin/go.d/docs/helper-packages.md`); what needs a boundary and67 what stays direct code. Any coupling across jobs or owners, any durable state, any scheduler or queue goes through68 the Architecture Gate below first.694. **Identity and lifecycle.** Name what survives a cycle, a restart, and a reload; which identity is stable and which70 is display metadata; which concurrent owners actually exist (the same job's old and new runtime is one case,71 different jobs another). Trace one successful cycle and one cycle with unfinished cleanup before designing the72 engine. If housekeeping would stop measurement, either state the real dependency or separate the two state73 dimensions; do not shorten a safety interval to hide the stall.745. **Operator surface.** One decision record row per proposed option, the mode form as a user task, and the75 consumer traces for defaults and null: `operator-surface.md`. List the implementation details you intentionally do76 not expose.776. **Metric semantics.** One measurement truth table row per new observation (below). Derive names, units, and help78 text from the table, never the other way round.797. **Evidence plan.** What proves the real path (real construction, real transitions, the shipped adapter), which80 fakes carry independent semantics, and what cannot be verified locally and is therefore stated as unverified. Test81 rules live in the V2 skill's Tests section.8283## Architecture Gate8485For configured vnode acquisition and named attachment, use86`src/go/plugin/framework/vnodes/README.md#ownership` and87`src/go/plugin/framework/vnodes/README.md#collector-attachment` as the existing ownership contract before proposing88collector-owned identity polling or shared connection settings.8990**When:** a proposal makes one job depend on another job's state (scanning its journals, waiting for its cleanup,91sharing an operational lock, consulting a registry for permission to run), introduces durable state for otherwise92independent reads, adds a scheduler or queue, lets cleanup freeze measurement, or builds a generic engine around one93provider's quirks. **Do:** answer the five questions in the design note before code; if the answer is "unknown",94investigate or ask, never implement a placeholder for later review. **Don't:** repair the proposal by making the scan95faster, raising a file cap, adding retries, or writing tests that expect the coupling; those preserve the wrong96dependency. **Evidence:** concrete object keys, owner identities, and a named invariant, not "there could be races".97**Boundary:** a genuinely shared resource may require coordination; the gate is not a ban on locks, shared clients,98or framework infrastructure, and "this looks complex" is not a finding without a named dependency and consequence.991001. **Shared resource.** What concrete object, namespace, limit, or external protocol is shared? Sharing a directory,101 an SDK, or a provider type is not a collision.1022. **Necessity.** Which supported execution fails with independent per-job ownership? Show the collision.1033. **Narrowest boundary.** Why is per-owner identity and exclusion insufficient? Compare with the proposal using real104 keys and owner identities.1054. **Failure propagation and cost.** Can a stopped, corrupt, or unreachable job block a healthy one? State the cost106 variables: work per job per call, per retained item, remote calls, state serialization, lock scope, growth with107 jobs and backlog. Use source-derived bounds at design time; a shipped hot-path change still follows108 `src/go/AGENTS.md` "Hot-Path And Benchmark Discipline". A cache that preserves the failure coupling is not a fix.1095. **Decision.** Necessary coordination is exposed as an operational trade-off and gets the applicable design110 approval. Unnecessary coordination is redesigned around the actual owner boundary. For durable ownership also111 state the recovery consequences: same owner versus different owner, label change, credential rotation, location112 change, rename. Isolation does not solve identity migration; say what a renamed job does and does not inherit.113114Worked example, from the S3check original: the proposal scanned every job's ownership files under a global handoff115lock before publishing a probe. Q1: object keys were already namespaced per Agent and per job, so no key collided;116a shared directory is not a shared resource. Q2: no supported execution failed with per-job ownership; the only real117overlap was the same job's old and new runtime during reload. Q3: a per-owner lock and journal (owner = Agent118registry ID + job name) covered that overlap. Q4: an unrelated corrupt journal blocked a healthy job, scanning was119O(jobs²), and a 256-file cap turned growth into a hard failure. Decision: reject the coordination, redesign around120owner identity, record that a renamed job does not adopt old ownership. All of this was decidable from the proposal.121122## Measurement Truth Table123124**When:** every new observation, and every change to what an existing one means. **Do:** fill one row per125observation, then map states to values, before writing metric names or `charts.yaml`. **Don't:** emit a value that126looks like a measurement for something not measured; a skipped operation has no duration, a failed attempt is not a lag127or success sample (its request duration may still be a valid measurement of the request), a waiting state is not a zero128(`.agents/skills/collectors-authoring/collector-practices.md#14-gaps-are-data`). **Evidence:** the table itself, plus a129test that drives each row's state through the real path and asserts emitted / omitted / retained. **Boundary:**130human-readable configuration does131not prohibit millisecond latency charts; the table decides units per chart.132133| Column | Meaning |134|---|---|135| Eligibility | when the observation may be produced at all |136| Start event | what starts the measurement (for S3check delete lag: the successful source DELETE, not the first attempt) |137| Stop event | what ends it (observed destination absence; a timeout ends the attempt, a breach does not) |138| Scope | per probe, per target, per job |139| Aggregation | how repeated calls become one value or status before chart labels collapse them |140| Consequence | what the operator should conclude from it |141142Then map each of: measured, missing, skipped, waiting, retrying, failed operation, failed collection, backpressured,143terminal, to one of: emitted value, omitted (gap), retained last terminal value, state-set state. Distinguish a144measured unhealthy target from an inability to collect; make sure the framework commits the intended failure145observations; check every early return against the table. Comparisons follow the wording ("exceeding" is strict).146147## Lifecycle Entry Points148149**When:** designing `Init`, `Check`, `Collect`, `Cleanup`, and any `Run`. **Do:** review every entry point, including150partial initialization, DynCfg `test`, autodetection, reload, and stop, not only `Collect` followed by a clean151shutdown. Decide, per entry point, what it may do, and record it; the V2 skill's Core Style owns the resulting rules152(`Check` detection-only, cleanup caller-cancelled or detached with a fixed budget independent of request and retry153settings; S3check chose five seconds). **Don't:** derive a shutdown budget from public tuning, or make orderly cleanup154the crash-recovery mechanism. **Evidence:** the trace per entry point in the design note; tests for cancellation and155partial init. **Boundary:** a read-only collector closing idle connections needs no journal or timeout machinery;156background contexts are not banned, unbounded ones are.157158## Simplification As Engineering159160**When:** throughout, and at the final pass. **Do:** prefer direct ownership, small types, clear transitions, and161existing helpers over generic engines and defensive layers; name files by responsibility and check that the content162matches; split along operations or state boundaries, not line counts; share only real semantics and keep distinct163provider logic distinct; run the V2 skill's Pre-PR final sweep before review. **Don't:** reject necessary state or164boundaries to minimize the diff, or add pooling and caching to satisfy a slogan. **Evidence:** per-cycle cost stated165from the source (a global O(N²) scan matters; a bounded map in a network-bound check does not). **Boundary:** file166length is a signal, not a limit; splitting one function into arbitrarily named helpers is not architecture.167168## Reading Sequence1691701. Select affected items from the task table. Implementation records its applicable note; review checks existing171 contracts and evidence under `./SKILL.md#review-and-implementation`.1722. Read `operator-surface.md` for option/mode decisions and `config-schema.md` for form authoring or review.173 Read `mutating-collectors.md` for remote mutation or durable local state, using its scoped routes.1743. For implementation mechanics or their review, use `src/go/plugin/go.d/docs/how-to-write-a-collector.md` and the V2175 skill when that framework applies; contract-preserving migrations use the migration guide.1764. When metadata or delivery is affected, use `.agents/skills/collectors-metadata-yaml/SKILL.md` and177 `.agents/skills/integrations-lifecycle/consistency.md`. Reading delivery guidance does not request regeneration.