Detection engineering standards
Criteria verified as of August 2026. Re-verify on the web before committing to anything (§8).
1. Scope and triggers
Applies to designing, writing, testing, deploying, measuring and retiring detection content:
detection backlog and hypotheses, data model and source onboarding, schema
normalisation (OCSF, ECS), rule authoring in any format (Sigma, YARA/YARA-X, KQL, SPL,
EQL, YARA-L, Suricata, Zeek, Wazuh, Falco), the detection-as-code pipeline (repo, CI,
tests, deployment, versioning), coverage map against MITRE ATT&CK, enrichment and
deduplication, thresholds and severity, false positive management, per-alert runbook,
adversarial validation of the detection, use of threat intelligence, and the programme's
metrics (MTTD, coverage per source, FP per rule, alerts per analyst per hour).
Triggers: Sigma *.yml with detection:/logsource:, sigma-cli/sigmac, pySigma,
*.yar/*.yara, yr (YARA-X), rules/**/*.toml from elastic/detection-rules,
detection_rules test, contentctl/contentctl-ng, panther_analysis_tool, suricata.yaml
and *.rules, *.zeek scripts, Wazuh local_rules.xml/decoders, falco_rules.local.yaml,
ATT&CK Navigator .json layers, chainsaw, hayabusa, atomic-red-team, "detection
rule", "SIEM use case", "false positive", "tuning", "ATT&CK coverage", "MTTD".
Guiding principle: detection is an engineering product with a lifecycle, not a
catalogue of bought alerts. It is governed by three invariants: (1) without the right source there is
no detection — the data model precedes the rule; (2) every rule is tested with the
real attack it claims to detect before it is believed; (3) an alert with no possible action is not
an alert, it is noise with an owner. Uncomfortable corollary: coverage ≠ detection. "We cover
300 techniques" almost always means "we have 300 rules nobody has run against an
attack".
Not applicable: see observability-standards (telemetry for diagnosing: OpenTelemetry,
Prometheus, Loki, cardinality, cost, Alertmanager — the boundary is the purpose, not the
tool: the same log feeds both, there to explain an outage, here to discover
an adversary; the pipeline, retention and integrity of the telemetry are theirs, the
analytical content is mine), sre-practice-standards (on-call, SLOs, error budget, burn rate,
postmortems), incident-management-standards (governance of the incident my alert
triggers: declaration, severity, IC, communication — my deliverable ends exactly at the
handoff, and its retro sends work back to me), incident-response-forensics-standards (technical
response and investigation: containment, imaging, timeline, eradication — a
bidirectional boundary: there they investigate what I detect, and every investigation must come back
turned into a new rule; the use of YARA for forensic hunting and evidence triage is
theirs, the authoring and governance of the rule is mine), offensive-security-standards (purple
team, adversary emulation, Atomic Red Team and Caldera as generators of the authorised
attack and its deconfliction with the SOC — they generate the activity, I write and validate
the detection), container-runtime-security-standards (which runtime signal matters —
unexpected shell, writes to binaries, runtime socket, drift— and the deployment of the
eBPF agent: the Falco rule is born there and is governed here, with my lifecycle, my tests
and my coverage), linux-hardening-standards and selinux-standards (CIS baseline, auditd and
SELinux AVCs as a source: configuring the audit ruleset is theirs, turning
those events into detection is mine), windows-server-ad-standards (which AD event matters and
why), networking-standards (segmentation, NetFlow/IPFIX and the network Suricata/Zeek
observe), appsec-standards (A09 Security Logging and Alerting Failures: which events
the application must emit — authn, denied authz, privilege change; I consume that
emission, I do not design it), vulnerability-management-standards (CVE, CVSS/EPSS/KEV and patching
SLAs), privacy-engineering-standards (PII inside security logs, minimisation
and retention limits by purpose), secrets-management-standards (custody and rotation of
secrets — shared boundary: a leaked secret or a secret used from an anomalous origin is a
detection case of mine, fed by their manager and scanning telemetry), grc-compliance-standards (the
regulatory control requiring monitoring and its evidence), identity-access-management-standards (IdP design,
conditional access policies and OAuth flows — I detect their abuse, I do not configure them),
threat-intelligence-standards (bidirectional boundary: the hypothesis, the indicator and its
expiry are theirs, the rule is mine — and the return of sightings is theirs: an indicator that
expires for lack of sightings is retired there, by their criteria, not here).
2. Default decisions
Data verified Aug 2026. Verify the latest version, licence and status on the web before
pinning it in a real project (§8): this ecosystem rotates in months and several projects
changed governance in 2025-2026.
| Area |
Default |
Justifiable alternative / vetoed |
| Portable authoring format |
Sigma — spec v2.1.0 (Sep 2025, formalises correlations and meta-rules), pySigma 1.5.0, sigma-cli 3.1.0, SigmaHQ/sigma rules r2026-07-01 |
Writing directly in the SIEM's language only when translation loses semantics (§3). Hard minimum: pySigma ≥ 1.3.0 — earlier versions executed code via template vars (today behind --enable-template-vars and a sandboxed Jinja2) |
| Files and memory |
YARA-X 1.19.0 |
Classic YARA 4.5.8 only for third-party engine compatibility: VirusTotal declared it in maintenance mode when publishing YARA-X 1.0.0 (Jun 2025) and new modules (macho, lnk) only reach YARA-X. Migrate while validating the ruleset (≈99 % compatible at rule level) |
| Threat reference framework |
MITRE ATT&CK v19.1 (Apr 2026) + Navigator 5.3.2 |
None. It is the lingua franca. Watch two breaks: v18.0 replaced Detections with Detection Strategies (DETxxxx) + Analytics (ANxxxx) and deprecated the legacy Data Sources; v19 split Defense Evasion into Stealth (TA0005) and Defense Impairment (TA0112) — use the official crosswalk, do not renumber by hand |
| Coverage measurement |
DeTT&CT 2.2.0 over Navigator layers, scoring data source quality, not just rule presence |
Your own spreadsheet (it desynchronises on its own). Vetoed: presenting "techniques with a rule" as coverage without scoring visibility |
| Adversarial validation |
Atomic Red Team (MIT, rolling model with no tags) for atomic tests per technique |
Apache Caldera for chains and campaign emulation — it changed governance: MITRE contributed it to the Apache Incubator on 20 May 2026, canonical repo apache/caldera. Hard minimum ≥ 5.1.0 because of CVE-2025-27364 (CVSS 10.0, preauth RCE in agent compilation) |
| Normalisation schema |
The one of the engine you consume, declared explicitly: ECS 9.4.0 in the Elastic/Sigma world; OCSF 1.8.0 (Linux Foundation since Nov 2024) in a data lake and for multi-vendor interoperability |
There is no single winner today. Frequent correction: ECS was not donated to OCSF, but to OpenTelemetry (Apr 2023), and that convergence is still incomplete. Pick one per platform and write it down; translate at the edge, not in the rule |
| Low-cost self-hosted SIEM |
Wazuh 4.14.7 (GPLv2) |
Hard minimum 4.14.4+. Wazuh has a serious history: CVE-2025-24016 (CVSS 9.9, deserialisation in DAPI, fix 4.9.1) was actively exploited by Mirai botnets. It is a server exposed to agents: treat it as Tier 0 |
| Commercial SIEM |
Whichever you already have; portability comes from Sigma, not the product |
Elastic Security 9.4.x (triple licence: AGPLv3 / SSPL / Elastic License — verify which applies to your deployment); Microsoft Sentinel; Google SecOps (formerly Chronicle) with YARA-L 2.0; Splunk ES |
| Network — IDS/IPS |
Suricata (8.0.6 and 7.0.17 in parallel, GPLv2) for signatures and protocol detection |
Zeek 8.0.9 (BSD) for network metadata and context: they do not compete, they complement each other. Zeek gives you the conn.log/ssl.log that makes the Suricata alert investigable |
| Container runtime |
Falco 0.44.1 (Apache-2.0, CNCF graduated since Feb 2024) |
The choice of agent and the what to watch belongs to container-runtime-security-standards; here the content lifecycle |
| Offline EVTX triage |
Hayabusa 3.10.0 (AGPL-3.0, ATT&CK v19) or Chainsaw 2.16.2 (GPL-3.0) — both run Sigma over evidence |
Manual EVTX parsing. Also useful as a test engine for Windows Sigma rules in CI |
| Third-party content as a base |
SigmaHQ (rules), elastic/detection-rules, splunk/security_content (ESCU 6.3.0, Apache-2.0), panther-analysis 3.112.0 (Apache-2.0), Nextron signature-base (DRL 1.1 licence — check before using in a commercial product) |
Consuming the whole package without calibrating (§7). Vetoed: closed vendor rules whose logic you can neither read nor version |
Projects not to choose in 2026 (verified): Matano — no commits since Jan 2025;
DetectionLab — declared dead by its author since 2023. Panther: Databricks announced an
acquisition agreement on 16 Jun 2026 and the release says nothing about the future of the OSS —
panther-analysis is still Apache-2.0 and active, but it is a bet with governance risk.
Worth mentioning in case it applies to the European public sector: OpenTIDE (European Commission, EUPL 1.2,
org OpenTideHQ) is today the most complete formal open-source detection-as-code framework.
3. Structure and conventions
3.1 The lifecycle (a rule is a software artifact, not a ticket)
idea/hypothesis → data source → rule → test → deployment by CI → measurement → tuning → retirement
- Idea/hypothesis: every rule is born from a written hypothesis (
"an attacker with valid credentials will register a new device in Entra after device code phishing"), with an
ATT&CK technique, a reference actor or campaign, and expected value. Without a hypothesis there is no
criterion for knowing whether the rule works or for retiring it.
- Source before rule (§3.2): if the event does not arrive, the rule is theatre.
- Rule: in the repo, with complete metadata (§3.4), reviewed in a PR by somebody other
than the author.
- Test: mandatory unit test (§4), plus adversarial validation before raising the severity.
- Deployment by CI: never by hand in the SIEM console. The console is read-only for
content; what is edited there is lost at the next sync or produces invisible
drift.
- Measurement: FP rate, firings, actions taken. A rule with no telemetry about itself
cannot be governed.
- Retirement: a rule that has fired nothing useful in its review window is deleted or
downgraded to informational, in the same PR that establishes this. The cost of a dead rule is
not zero: it consumes compute, dirties the coverage and wears down the analyst who triages it.
Reference repo structure (adapt it, but have one):
detections/
rules/<platform>/<tactic>/<id>.yml # Sigma or the engine's native format
pipelines/<platform>.yml # pySigma processing pipelines
tests/<id>/{positive,negative}.jsonl # events that MUST and MUST NOT fire
runbooks/<id>.md # mandatory (§3.5)
coverage/navigator-*.json # generated layers, not hand-edited
deprecated/ # with reason and date, not silent deletion
3.2 Data model and telemetry: the part almost nobody does
- Onboarding a source is a project, not a checkbox. For every source:
which concrete events (by ID/category, not "the AD logs"), how they are transported,
how they are normalised, how long they are retained, what they cost and which detections they enable.
If it enables none, it is not ingested.
- Cost and retention are first-class design constraints, exactly as in
observability-standards. Healthy pattern: short hot retention (interactive search) +
long, cheap cold retention (investigation and compliance). Practical rule: the cold
window must cover the dwell time you assume, not the one you would like.
- Normalise at the edge, not in the rule. A rule that slices strings to reconstruct a
field the pipeline should have normalised is debt paid on every new source.
- Measure source health as if it were an SLI: ingestion latency, volume per hour
with an expected band, and alert on the absence of events — a source's silence is
indistinguishable from "nothing happened", and it is exactly what an attacker who
disables the agent produces. Detection of missing detection is the first rule you write.
- Integrity: security logs are kept separate from operational telemetry (different access
control, retention and immutability). The SIEM is the record an attacker
will want to erase.
Source priority today (by value/cost ratio, not by tradition):
- Identity and the cloud control plane — it is the main surface (§3.7).
- Endpoint/EDR (process, command line, parent/child, module loading).
- Authentication and directory (AD, Entra, IdP).
- CI/CD and the artifact registry (§5).
- Network (DNS, proxy/egress,
conn.log): indispensable for investigating, mediocre for
detecting on its own.
3.3 ATT&CK coverage with honesty
- Coverage is scored by visibility quality × detection quality, not by
rule count. Minimum scale per technique:
no telemetry → telemetry with no rule →
unvalidated rule → rule validated against a real execution → rule validated and with a response procedure. Only the last two count as coverage.
- A technique covered by a single variant is not covered. ATT&CK enumerates
behaviours, not commands: if the rule only matches the binary the atomic test used, you have
covered the tool, not the technique.
- Prioritise by relevant threat (adversary profile, sector, real surface), not by
filling the matrix. A uniformly green matrix is a sign the colour has been optimised.
- Generate the Navigator layers from the repo in CI. A hand-edited coverage map
lies as soon as the next PR is merged.
- When jumping ATT&CK versions, use the official crosswalk (the split of Defense
Evasion in v19 invalidates previous mappings) and treat the migration as a change with a PR and
review, not as a
sed.
3.4 Anatomy of a rule that can be governed
Mandatory metadata, no exceptions: a stable id (UUID, never reused), an actionable
title, status (experimental → test → stable → deprecated), a description with
the hypothesis, author/owner (a team, not a person), date/modified, a justified level,
ATT&CK tags with the framework version, falsepositives populated with real observed
cases, references and a link to the runbook.
- Severity is earned, not declared. A rule is born in
experimental with informational
severity; it goes up only with data: N days in production, a measured FP rate, and
adversarial validation passed. Raising the severity "because the technique is serious" is the direct route to
alert fatigue.
- Actionable context in the event itself: who (a resolved identity, not a GUID), which
asset (with owner and criticality), what happened (relevant raw command/fields), what is expected
of the analyst. An alert that forces you to open four consoles to know whether it matters has already
failed.
- Enrichment in the pipeline, not in the analyst's head: asset inventory,
criticality, owner, geolocation, reputation, and whether that activity was
authorised (change window, deconflicted red team exercise).
- Deduplication and grouping: the unit of work is the incident, not the alert. It is
grouped by the entities that define one and the same case (identity, host, session, campaign) with
a time window. Sigma v2 correlations (
event_count, value_count,
temporal) are the portable mechanism for expressing chains and thresholds without tying yourself to the engine.
- Portability with a declared limit: Sigma is the authoring format; the backend translates.
When the semantics do not survive translation (complex correlation, joins, engine-specific
functions, sequence analysis), write it in the native language and declare
the exception in the rule — do not force a Sigma that translates into something other than what it says.
- Freshness of pySigma backends: verify it, do not trust the label. The
official directory marks abandoned backends as stable (
sentinelone unpublished
since 2023, ibm-qradar-aql and insightidr equally behind). Before betting a
platform on a backend, look at the date of the last release, not the badge.
3.5 Mandatory runbook: a rule with no action is not deployed
Every alert carries a runbook versioned in the same repo and in the same PR, with: what the
firing means, how it is verified in 5 minutes (concrete queries, not "investigate"), what is
known benign, what immediate action is appropriate (contain, revoke session, isolate,
escalate), to whom it is escalated and at what severity, and which data to preserve before touching anything
(preservation belongs to incident-response-forensics-standards; the link is mandatory).
If you cannot write the runbook, you do not know what the rule detects: it is not merged.
3.6 Adversarial validation and regression detection
- No rule leaves
experimental without having fired against a real execution of the
technique. The attack generator belongs to offensive-security-standards; the evidence of
the firing is my deliverable.
- Purple team ≠ red team: the value is not "they got in", it is the executed
vs. detected vs. alerted vs. responded matrix by technique. Every undetected execution opens
a backlog item with an owner and a date.
- Regressions: a rule that used to work stops working on its own when the agent changes,
or the vendor's schema, the OS version or a binary's path. Scheduled and automatic
execution of a subset of atomics against a controlled environment, with an alert if the expected
detection does not appear. A real and recent example of why this is not theoretical: Defender's
advanced hunting schema renamed
AADSignInEventsBeta to
EntraIdSignInEvents and removed the legacy tables on 9 Dec 2025, and on 25 Feb 2026 the
booleans went from 1/0 to True/False — each of those changes breaks rules
silently.
- Every adversarial execution is deconflicted with the SOC beforehand: an exercise that consumes the
real incident process is a process failure, not a test passed.
3.7 Identity and cloud: where the detection that matters lives today
The perimeter is identity. The minimum sources, with exact names (verify §8):
- Entra ID:
SigninLogs (interactive), AADNonInteractiveUserSignInLogs,
AADServicePrincipalSignInLogs, AADManagedIdentitySignInLogs, AuditLogs and
MicrosoftGraphActivityLogs (requires P1/P2). In 2026 Agent logs were added for
Entra Agent ID, and AADNonInteractiveUserSignInLogs now carries an Agent column: AI
agent identities are a new surface and must be ingested from day one.
- AWS CloudTrail: there are now four categories, not two — Management, Data, Network
activity (
eventCategory = NetworkActivity; the only valid errorCode is
VpceAccessDenied, useful for seeing credentials outside your organisation using your VPC
endpoints) and Insights. Only management is on by default: data and network events
are enabled explicitly and budgeted for.
- GCP:
cloudaudit.googleapis.com%2Factivity (Admin Activity, always),
%2Fdata_access (disabled by default except for BigQuery), %2Fsystem_event,
%2Fpolicy. For consent abuse in Workspace, the key log is the OAuth Token
Audit (applicationName=token, authorize/revoke events, with scope_data,
client_id, app_name); default retention 6 months — if your investigation
window is longer, export it.
- GuardDuty Extended Threat Detection is on by default and at no extra cost, with a rolling
24 h window. Verified operational trap: it ignores archived findings,
including those archived by your own suppression rules — aggressive tuning blinds you to
attack sequence correlation.
Mandatory identity use cases (defensible ATT&CK chain:
T1566.002 → T1528 → T1550.001 → T1098.005 / T1098.001; there is no dedicated sub-technique for
device code phishing — the one that mentions it explicitly is T1566.002, not T1528):
- Device code phishing: the decisive field in Entra is
AuthenticationProtocol == "deviceCode".
It is the technique of Storm-2372 (Microsoft, Feb 2025; reclassified on 31 Jul 2026 as a
sub-cluster of Midnight Blizzard/SVR), which pivoted to the client ID
29d9ed98-a469-4536-ade2-f981bc1d605e (Microsoft Authentication Broker) to obtain a
PRT and register a device. Preventive control: the Conditional Access "Authentication flows"
condition. Planning notice: Microsoft is rolling out a managed policy
"Block device code flow" which it creates in report-only and enables on its own after a minimum of 45
days, warning 28 days in advance; audit it by filtering AuditLogs on the initiator
Microsoft Managed Policy Manager. Critical caveat: a CA policy targeting users
does not cover service principals.
- Device registration and new credential: adding or changing credentials on
service principals and app registrations. It is exactly the pattern of the
Commvault/Metallic case (CISA, May 2025, CVE-2025-3928 in KEV): theft of client secrets
from a cross-tenant app.
- Abuse of a third-party integration's OAuth token: the Salesloft Drift / UNC6395 case
(Aug 2025) exfiltrated Salesforce data with legitimate OAuth tokens, searching with mass
SOQL for strings such as
AKIA, Snowflake, password, secret, and deleting the records of
the query jobs. Signals: Salesforce Event Monitoring UniqueQuery events, UA
Salesforce-Multi-Org-Fetcher/1.0. Design lesson: a third-party app's consent
is a permanent key with no MFA — inventory and watch consents the way you
watch administrators.
- Impossible travel and session anomaly: useful, but only as enrichment or a low-severity
signal. In a workforce with a corporate VPN and mobiles it generates massive FPs; never
as a paging alert on its own.
- Graph↔sign-in correlation (verified canonical pattern):
MicrosoftGraphActivityLogs | join ... on $left.SignInActivityId == $right.UniqueTokenIdentifier
— it is what turns "a token did things" into "this session, of this user, from this
origin, did these things".
3.8 Threat intelligence: pyramid of pain, not shopping list
- IoCs expire; TTPs do not. Hashes and IPs are rotated in hours and their value is almost purely
retrospective; tools and TTPs cost the adversary and survive months or years. The
engineering effort goes to the top of the pyramid; IoCs are automated and forgotten.
- Every IoC comes in with an expiry and a provenance. A feed with no expiry date is a
source of FPs that grows on its own. Indiscriminate free feeds generate more triage than they
save: they are measured like any rule (§6) and cut if they do not pay off.
- Systematic retro-hunt: every new IoC is searched backwards in the cold retention, not
only forwards. That is where the IoC does earn its keep.
- The ultimate goal of TI is not the list: it is prioritising the detection backlog by
relevant adversary.
4. Quality gates (they break the build)
In increasing order of cost. The first four are blocking on every detection PR.
- Lint and schema: the rule parses, validates against the format's schema (Sigma spec 2.1,
detection-rules TOML, the engine's schema) and compiles with the target backend
(sigma convert). A rule that does not translate does not exist.
- Complete metadata: a unique, non-reused
id, owner, status, ATT&CK tags
existing in the declared version of the framework, non-empty falsepositives.
- Runbook present and linked. No runbook, no merge (§3.5). Non-negotiable.
- Unit test with positives and negatives: at least one event that must fire and
several that must not (the rule's own known FPs). Valid engines:
detection_rules test (Elastic), panther_analysis_tool test, Hayabusa/Chainsaw over
sample EVTX, or the backend's runner. A rule with no negative test is not tested:
it is declared.
- Schema non-regression test: the test events are validated against the current
schema (ECS/OCSF/vendor table), so that a field rename breaks the build
and not the detection in silence (§3.6).
- Coverage: the Navigator layer is regenerated in CI and the diff is reviewed. Adding a
rule that does not move the declared coverage is a sign of a duplicate.
- Adversarial validation before promoting to
stable or raising severity: a real
atomic execution and evidence of the firing attached to the PR.
- Deployment by pipeline with versioning and rollback:
Sentinel repositories (GA
Mar 2026), elastic/detection-rules with a version lock per stack branch, or the equivalent
mechanism. Manual deployment in the console is forbidden.
Measurable promotion rule (adjust the numbers, but fix some): experimental → test
after compiling and passing tests; test → stable after ≥ 2 weeks in production with a measured
FP rate < 20 % and adversarial validation passed. It is measured before raising the severity,
never after.
5. Security of the detection programme itself
- The supply chain of security tooling is a first-order target, and 2026
proved it. The compromise of Trivy by the actor TeamPCP (CVE-2026-33634, CVSS
9.4, in CISA KEV since 26 Mar 2026) went as far as force-pushing 76 of 77 tags of
trivy-action and all 7 of setup-trivy, with a payload that read the memory of the
Runner.Worker process to extract masked secrets; the same campaign hijacked the 35 tags
of Checkmarx/kics-github-action. Analogous precedent: tj-actions/changed-files
(CVE-2025-30066). Operational consequences, without exception:
- Pin by commit SHA every GitHub Action in the detection pipeline; pin by
digest every image; verify signatures and checksums of the detection binaries.
- Immutable releases enabled in your own content repos.
- Do not confuse provenance with goodness: the CanisterWorm variant went as far as generating
valid SLSA Build L3 attestations via Sigstore for malicious packages. The signature
proves who built it, not what it does.
- The detail of pipeline hardening belongs to
cicd-standards; here it is an entry
requirement.
- Verified Aug 2026: none of the recommended detection tools (Sigma,
YARA/YARA-X, Atomic Red Team, Caldera, Falco, Suricata, Zeek, Wazuh, Elastic) appears as a
victim of a repository compromise in 2025-2026. The vector, however, applies to them
just the same: do not treat it as ruled out, but as not having happened yet.
- The SIEM and its rules are sensitive data. The ruleset reveals exactly what you see and what
you do not: access control on the detection repo, review of who can disable or modify
rules, and an alert on the modification or silencing of the content itself. Disabling
a rule is a security event.
- Never embed secrets in rules or in queries (API tokens, enrichment
credentials, webhook keys). They go to the secrets manager
(
secrets-management-standards).
- PII and minimisation: security logs contain personal data by definition.
Legal basis, retention by purpose and access control belong to
privacy-engineering-standards; the requirement here is that the rule and its enrichment do not
widen the set of personal data beyond what is needed to decide.
- Integrity and non-repudiation of the evidence: append-only logs, a synchronised and
verified clock (clock drift destroys any temporal correlation and any
forensic timeline) and control over who can purge.
6. Operability, cost and programme metrics
Metrics that are published (and what corrupts them)
| Metric |
What it measures |
How it gets corrupted |
| MTTD |
Time from the adversary's activity to detection |
Measuring from ingestion, not from the event; excluding what was never detected (survivorship bias). It is only honest with purple team, where you know the real t0 |
| Coverage per source |
% of assets of each class reporting the expected source |
Counting configured sources instead of sources that are sending events now |
| FP rate per rule |
Firings closed as benign / total firings |
Not recording the analyst's disposition; then there is no data and tuning is opinion |
| Alerts per analyst per hour |
Real triage load |
Counting alerts instead of grouped incidents. An alarm threshold for the programme, not for the shift |
| Orphaned rules |
No owner, no firings, no runbook, no review within its window |
None: it is the metric hardest to dress up and the one that best predicts the programme's collapse |
| % of adversarially validated rules |
Real coverage |
Counting written rules |
- Alert fatigue is the domain's modal failure. If the analyst ignores the queue,
detection does not exist even if the rules are perfect. Work priority when the queue
overflows: reduce noise before adding coverage, always.
- Tuning ≠ disabling. The correct order is: (1) fix the data or the enrichment,
(2) narrow the rule's condition, (3) a narrow, documented exception with an
expiry, (4) downgrade to informational, (5) retire. A broad and permanent suppression is
a detection hole shaped like tuning — and on AWS it also blinds GuardDuty's
correlation (§3.7).
- Cost: the SIEM is paid for by ingestion, by retention or by query compute depending on the
product — find out your billing unit before designing, because it decides what to optimise.
Levers in order: do not ingest what does not enable detection; filter and aggregate at the edge;
tiered retention to cheap storage; efficient scheduled queries. Ingesting "just in
case" is the most expensive way of detecting nothing.
- Rule performance: scheduled queries with a coherent window and frequency
(window ≥ frequency + ingestion latency, or you lose events at the edge), no joins
over the whole history, no catastrophic regexes. A rule that does not finish within its
interval does not detect: it skips executions.
- Platform change with a hard date: if you use Microsoft Sentinel, the product is already
GA in the Defender portal (Jan 2026) and stops being supported in the Azure portal
after 31 Mar 2027. It is a migration project with a date, not a UI change.
7. Sustainability and prohibitions
- Cadence: review ATT&CK and remap on every major framework release (~half-yearly;
v18 and v19 brought structural breaks); review the entire ruleset quarterly with
explicit pruning; update
pySigma/backends and the community ruleset monthly;
review sources and their health continuously.
- Every exception and every suppression carries an expiry and an owner. Without a date, they become
permanent holes nobody remembers opening.
- Third-party content is calibrated on adoption, not "some day": it enters the repo, passes
the gates of §4 and is assigned an owner, or it does not enter.
- When a project changes governance (Caldera → Apache Incubator; Panther → Databricks),
review it as a platform risk, not as a press release: licence, release cadence
and who responds to a CVE.
FORBIDDEN
- ❌ Deploying a rule without a unit test (positive and negative).
- ❌ Deploying a rule without a runbook with a concrete action.
- ❌ Raising the severity without having measured the FP rate in production.
- ❌ Creating or editing content directly in the SIEM console, with no repo and no PR.
- ❌ Enabling the product's default rule pack without calibrating — the domain's central
antipattern: a SIEM full of rules nobody has touched is not coverage, it is a
noise queue that guarantees the good alert goes unnoticed.
- ❌ Presenting a count of rules or techniques as coverage without scoring visibility and
validation.
- ❌ Rules with no owner, no stable
id, no ATT&CK version tags or with empty
falsepositives.
- ❌ Writing rules for a source not confirmed in ingestion, or leaving the absence of events
from a source without an alert.
- ❌ Broad, permanent or ownerless suppressions; silencing a rule instead of fixing it.
- ❌ Alerting on IoCs with no expiry or provenance, or building the programme on IoCs instead
of on TTPs.
- ❌ Impossible travel (or other pure anomaly heuristics) as a standalone paging alert.
- ❌ Consuming GitHub Actions or images in the detection pipeline by mutable tag instead of
by SHA/digest, or treating an SLSA attestation as proof that the artifact is benign.
- ❌ Running adversarial validation without prior deconfliction with the SOC.
- ❌ Secrets or credentials embedded in rules, queries or enrichment pipelines.
- ❌ Starting anything new on Matano or DetectionLab (unmaintained), or on classic
YARA when YARA-X is available.
- ❌ Running Wazuh or Caldera on versions earlier than the minimums of §2 (critical CVEs, one
of them mass-exploited).
- ❌ Forcing into Sigma a logic the backend does not support and trusting the
result.
8. Mandatory web verification
Before pinning any version, licence, project status or table/field name,
look it up — do not recall it. Data verified Aug 2026 (it expires fast): Sigma spec
v2.1.0, pySigma 1.5.0, sigma-cli 3.1.0, SigmaHQ rules r2026-07-01; ATT&CK
v19.1 (Apr 2026) and Navigator 5.3.2; ATT&CK Workbench 4.10.0; OCSF 1.8.0 (LF
since Nov 2024); ECS 9.4.0; Elasticsearch 9.3.0 / Elastic Security 9.4.4; Wazuh
4.14.7; Suricata 8.0.6 and 7.0.17; Zeek 8.0.9; Falco 0.44.1 (CNCF graduated
Feb 2024); YARA-X 1.19.0 and YARA 4.5.8; Splunk ESCU 6.3.0; panther-analysis
3.112.0; Chainsaw 2.16.2; Hayabusa 3.10.0; DeTT&CT 2.2.0.
- The current ATT&CK version and its changelog, including the crosswalk for the split of
Defense Evasion (v19) and the Detection Strategies/Analytics model (v18) — before
remapping anything.
- The real freshness of the pySigma backend you are going to use: the date of the last release in the
pySigma-plugin-directory, not the stable badge.
- Governance and health of Caldera (Apache Incubator since May 2026, current version not
verified here) and of Panther (acquisition by Databricks announced Jun 2026, future
of the OSS not declared).
- Exact names of the vendor's tables, fields and logs before writing the query:
the Defender/Entra, CloudTrail and Cloud Audit Logs schemas changed in 2025-2026 and
keep changing.
- Hard platform dates: end of support for Sentinel in the Azure portal
(31 Mar 2027, verify), status of the content migrations under way.
- Advisories and CVEs for your SIEM and your agents (Wazuh and Elastic have had recent
criticals) and for the tooling you run in CI.
- Supply chain compromises of any new action, image or binary in the
detection pipeline before adopting it (§5).
Declared gaps — not verified in this document, verify them yourself before using them:
- The current version of Apache Caldera (incubating).
- The version of Splunk Enterprise Security and product news after the acquisition by
Cisco (the official documentation was not accessible during verification).
- Formal EOL dates for the Suricata 7.0 and 8.0 branches, and whether a 9.0 release exists or is
planned (the upgrade guide mentions it; no release has been published).
- The status of OCSF support in Elastic Security and in Splunk in 2026.
- Whether Red Canary (maintainer of Atomic Red Team) is today part of Zscaler — a secondary
source, unconfirmed against a primary release.
- GA dates for the Conditional Access "Authentication flows" condition, for CloudTrail's
network activity events and for GuardDuty Extended Threat Detection.
- Secondary ATT&CK IDs cited in identity literature (T1621, T1539, T1550.004,
T1078.004, T1098.002/.003, T1606.002): not individually verified against the site.
- Whether the recommended detection projects consumed
trivy-action or
tj-actions/changed-files during their exposure windows: not ruled out, just not
found.
If the web contradicts this document, the web wins — flag the discrepancy.
1---2name: detection-engineering-standards3description: Security detection engineering — building and governing SIEM detections as code. Use when writing or tuning Sigma rules (sigma-cli, pySigma, sigma correlations), YARA or YARA-X rules, Elastic detection-rules TOML, KQL, SPL, EQL or YARA-L analytics, Suricata and Zeek signatures, Wazuh decoders and rules, ATT&CK Navigator coverage layers, DeTT&CT, Chainsaw or Hayabusa triage, alert runbooks and false-positive rates, detection unit tests in CI, OCSF or ECS log normalization, or onboarding a new log source into a SIEM.4metadata:5 source: https://skillmd.com/skills/serialexperimentslainnnn/detection-engineering-standards6---78# Detection engineering standards910Criteria verified as of **August 2026**. Re-verify on the web before committing to anything (§8).1112## 1. Scope and triggers1314Applies to designing, writing, testing, deploying, measuring and **retiring** detection content:15detection backlog and hypotheses, data model and source onboarding, schema16normalisation (OCSF, ECS), rule authoring in any format (Sigma, YARA/YARA-X, KQL, SPL,17EQL, YARA-L, Suricata, Zeek, Wazuh, Falco), the detection-as-code pipeline (repo, CI,18tests, deployment, versioning), coverage map against MITRE ATT&CK, enrichment and19deduplication, thresholds and severity, false positive management, per-alert runbook,20adversarial validation of the detection, use of threat intelligence, and the programme's21metrics (MTTD, coverage per source, FP per rule, alerts per analyst per hour).2223Triggers: Sigma `*.yml` with `detection:`/`logsource:`, `sigma-cli`/`sigmac`, `pySigma`,24`*.yar`/`*.yara`, `yr` (YARA-X), `rules/**/*.toml` from `elastic/detection-rules`,25`detection_rules test`, `contentctl`/`contentctl-ng`, `panther_analysis_tool`, `suricata.yaml`26and `*.rules`, `*.zeek` scripts, Wazuh `local_rules.xml`/`decoders`, `falco_rules.local.yaml`,27ATT&CK Navigator `.json` layers, `chainsaw`, `hayabusa`, `atomic-red-team`, "detection28rule", "SIEM use case", "false positive", "tuning", "ATT&CK coverage", "MTTD".2930**Guiding principle**: detection is an **engineering product with a lifecycle**, not a31catalogue of bought alerts. It is governed by three invariants: **(1) without the right source there is32no detection** — the data model precedes the rule; **(2) every rule is tested with the33real attack it claims to detect** before it is believed; **(3) an alert with no possible action is not34an alert, it is noise with an owner**. Uncomfortable corollary: **coverage ≠ detection**. "We cover35300 techniques" almost always means "we have 300 rules nobody has run against an36attack".3738**Not applicable**: see `observability-standards` (telemetry for **diagnosing**: OpenTelemetry,39Prometheus, Loki, cardinality, cost, Alertmanager — **the boundary is the purpose, not the40tool**: the same log feeds both, there to explain an outage, here to discover41an adversary; the pipeline, retention and integrity of the telemetry are theirs, the42analytical content is mine), `sre-practice-standards` (on-call, SLOs, error budget, burn rate,43postmortems), `incident-management-standards` (**governance of the incident my alert44triggers**: declaration, severity, IC, communication — my deliverable ends exactly at the45handoff, and its retro sends work back to me), `incident-response-forensics-standards` (**technical46response and investigation**: containment, imaging, timeline, eradication — a47**bidirectional** boundary: there they investigate what I detect, and **every investigation must come back48turned into a new rule**; the use of YARA for forensic *hunting* and evidence triage is49theirs, the **authoring and governance of the rule** is mine), `offensive-security-standards` (purple50team, adversary emulation, Atomic Red Team and Caldera as **generators** of the authorised51attack and its deconfliction with the SOC — **they generate the activity, I write and validate52the detection**), `container-runtime-security-standards` (**which runtime signal matters** —53unexpected shell, writes to binaries, runtime socket, drift— and the deployment of the54eBPF agent: **the Falco rule is born there and is governed here**, with my lifecycle, my tests55and my coverage), `linux-hardening-standards` and `selinux-standards` (CIS baseline, `auditd` and56SELinux AVCs **as a source**: configuring the audit ruleset is theirs, turning57those events into detection is mine), `windows-server-ad-standards` (which AD event matters and58why), `networking-standards` (segmentation, NetFlow/IPFIX and the network Suricata/Zeek59observe), `appsec-standards` (**A09 Security Logging and Alerting Failures**: which events60the application must **emit** — authn, denied authz, privilege change; I consume that61emission, I do not design it), `vulnerability-management-standards` (CVE, CVSS/EPSS/KEV and patching62SLAs), `privacy-engineering-standards` (PII inside security logs, minimisation63and retention limits by purpose), `secrets-management-standards` (custody and rotation of64secrets — **shared boundary**: a leaked secret or a secret used from an anomalous origin is a65**detection case of mine**, fed by their manager and scanning telemetry), `grc-compliance-standards` (the66regulatory control requiring monitoring and its evidence), `identity-access-management-standards` (IdP design,67conditional access policies and OAuth flows — I detect their abuse, I do not configure them),68`threat-intelligence-standards` (**bidirectional boundary**: the hypothesis, the indicator and its69expiry are theirs, the rule is mine — and **the return of *sightings* is theirs**: an indicator that70expires for lack of sightings is retired there, by their criteria, not here).7172## 2. Default decisions7374> Data verified Aug 2026. **Verify the latest version, licence and status on the web before75> pinning it in a real project (§8)**: this ecosystem rotates in months and several projects76> changed governance in 2025-2026.7778| Area | Default | Justifiable alternative / vetoed |79|---|---|---|80| Portable authoring format | **Sigma** — spec **v2.1.0** (Sep 2025, formalises *correlations* and *meta-rules*), `pySigma` **1.5.0**, `sigma-cli` **3.1.0**, `SigmaHQ/sigma` rules **r2026-07-01** | Writing directly in the SIEM's language **only** when translation loses semantics (§3). **Hard minimum: pySigma ≥ 1.3.0** — earlier versions executed code via *template vars* (today behind `--enable-template-vars` and a sandboxed Jinja2) |81| Files and memory | **YARA-X 1.19.0** | Classic YARA **4.5.8** only for third-party engine compatibility: VirusTotal declared it in **maintenance mode** when publishing YARA-X 1.0.0 (Jun 2025) and new modules (`macho`, `lnk`) only reach YARA-X. Migrate while validating the ruleset (≈99 % compatible at rule level) |82| Threat reference framework | **MITRE ATT&CK v19.1** (Apr 2026) + **Navigator 5.3.2** | None. It is the lingua franca. Watch two breaks: **v18.0** replaced *Detections* with **Detection Strategies (`DETxxxx`) + Analytics (`ANxxxx`)** and deprecated the legacy *Data Sources*; **v19** split **Defense Evasion** into **Stealth (TA0005)** and **Defense Impairment (TA0112)** — use the official *crosswalk*, do not renumber by hand |83| Coverage measurement | **DeTT&CT 2.2.0** over Navigator layers, scoring **data source quality**, not just rule presence | Your own spreadsheet (it desynchronises on its own). **Vetoed**: presenting "techniques with a rule" as coverage without scoring visibility |84| Adversarial validation | **Atomic Red Team** (MIT, *rolling* model with no tags) for atomic tests per technique | **Apache Caldera** for chains and campaign emulation — **it changed governance: MITRE contributed it to the Apache Incubator on 20 May 2026**, canonical repo `apache/caldera`. **Hard minimum ≥ 5.1.0** because of **CVE-2025-27364** (CVSS 10.0, preauth RCE in agent compilation) |85| Normalisation schema | **The one of the engine you consume**, declared explicitly: **ECS 9.4.0** in the Elastic/Sigma world; **OCSF 1.8.0** (Linux Foundation since Nov 2024) in a data lake and for multi-vendor interoperability | **There is no single winner today.** Frequent correction: **ECS was not donated to OCSF, but to OpenTelemetry (Apr 2023)**, and that convergence is still **incomplete**. Pick one per platform and write it down; translate at the edge, not in the rule |86| Low-cost self-hosted SIEM | **Wazuh 4.14.7** (GPLv2) | **Hard minimum 4.14.4+**. Wazuh has a serious history: **CVE-2025-24016** (CVSS 9.9, deserialisation in DAPI, fix 4.9.1) was **actively exploited by Mirai botnets**. It is a server exposed to agents: treat it as Tier 0 |87| Commercial SIEM | Whichever you already have; portability comes from Sigma, not the product | **Elastic Security 9.4.x** (**triple** licence: AGPLv3 / SSPL / Elastic License — verify which applies to your deployment); **Microsoft Sentinel**; **Google SecOps** (formerly Chronicle) with **YARA-L 2.0**; **Splunk ES** |88| Network — IDS/IPS | **Suricata** (8.0.6 and 7.0.17 in parallel, GPLv2) for signatures and protocol detection | **Zeek 8.0.9** (BSD) for network **metadata and context**: they do not compete, they complement each other. Zeek gives you the `conn.log`/`ssl.log` that makes the Suricata alert investigable |89| Container runtime | **Falco 0.44.1** (Apache-2.0, CNCF **graduated** since Feb 2024) | The choice of agent and the *what to watch* belongs to `container-runtime-security-standards`; here the content lifecycle |90| Offline EVTX triage | **Hayabusa 3.10.0** (AGPL-3.0, ATT&CK v19) or **Chainsaw 2.16.2** (GPL-3.0) — both run Sigma over evidence | Manual EVTX parsing. Also useful as a **test engine** for Windows Sigma rules in CI |91| Third-party content as a base | **SigmaHQ** (rules), **elastic/detection-rules**, **splunk/security_content (ESCU 6.3.0, Apache-2.0)**, **panther-analysis 3.112.0** (Apache-2.0), **Nextron `signature-base`** (**DRL 1.1** licence — check before using in a commercial product) | Consuming the whole package without calibrating (§7). **Vetoed**: closed vendor rules whose logic you can neither read nor version |9293**Projects not to choose in 2026** (verified): **Matano** — no commits since Jan 2025;94**DetectionLab** — declared dead by its author since 2023. **Panther**: Databricks announced an95acquisition agreement on 16 Jun 2026 and **the release says nothing about the future of the OSS** —96`panther-analysis` is still Apache-2.0 and active, but it is a bet with governance risk.97Worth mentioning in case it applies to the European public sector: **OpenTIDE** (European Commission, EUPL 1.2,98org `OpenTideHQ`) is today the most complete formal open-source detection-as-code framework.99100## 3. Structure and conventions101102### 3.1 The lifecycle (a rule is a software artifact, not a ticket)103104`idea/hypothesis → data source → rule → test → deployment by CI → measurement → tuning → retirement`105106- **Idea/hypothesis**: every rule is born from a written hypothesis (`"an attacker with valid107 credentials will register a new device in Entra after device code phishing"`), with an108 ATT&CK technique, a reference actor or campaign, and **expected value**. Without a hypothesis there is no109 criterion for knowing whether the rule works or for retiring it.110- **Source before rule** (§3.2): if the event does not arrive, the rule is theatre.111- **Rule**: in the repo, with complete metadata (§3.4), reviewed in a PR by somebody other112 than the author.113- **Test**: mandatory unit test (§4), plus adversarial validation before raising the severity.114- **Deployment by CI**: never by hand in the SIEM console. The console is *read-only* for115 content; what is edited there is lost at the next sync or produces invisible116 *drift*.117- **Measurement**: FP rate, firings, actions taken. A rule with no telemetry about itself118 cannot be governed.119- **Retirement**: a rule that has fired nothing useful in its review window **is deleted or120 downgraded to informational**, in the same PR that establishes this. The cost of a dead rule is121 not zero: it consumes compute, dirties the coverage and wears down the analyst who triages it.122123**Reference repo structure** (adapt it, but have one):124125```126detections/127 rules/<platform>/<tactic>/<id>.yml # Sigma or the engine's native format128 pipelines/<platform>.yml # pySigma processing pipelines129 tests/<id>/{positive,negative}.jsonl # events that MUST and MUST NOT fire130 runbooks/<id>.md # mandatory (§3.5)131 coverage/navigator-*.json # generated layers, not hand-edited132 deprecated/ # with reason and date, not silent deletion133```134135### 3.2 Data model and telemetry: the part almost nobody does136137- **Onboarding a source is a project, not a checkbox.** For every source:138 *which concrete events* (by ID/category, not "the AD logs"), *how they are transported*,139 *how they are normalised*, *how long they are retained*, *what they cost* and *which detections they enable*.140 If it enables none, it is not ingested.141- **Cost and retention are first-class design constraints**, exactly as in142 `observability-standards`. Healthy pattern: **short hot retention (interactive search) +143 long, cheap cold retention (investigation and compliance)**. Practical rule: the cold144 window must cover the *dwell time* you assume, not the one you would like.145- **Normalise at the edge, not in the rule.** A rule that slices strings to reconstruct a146 field the pipeline should have normalised is debt paid on every new source.147- **Measure source health as if it were an SLI**: ingestion latency, volume per hour148 with an expected band, and **alert on the absence of events** — a source's silence is149 indistinguishable from "nothing happened", and it is exactly what an attacker who150 disables the agent produces. *Detection of missing detection* is the first rule you write.151- **Integrity**: security logs are kept separate from operational telemetry (different access152 control, retention and **immutability**). The SIEM is the record an attacker153 will want to erase.154155**Source priority today** (by value/cost ratio, not by tradition):1561. **Identity and the cloud control plane** — it is the main surface (§3.7).1572. **Endpoint/EDR** (process, command line, parent/child, module loading).1583. **Authentication and directory** (AD, Entra, IdP).1594. **CI/CD and the artifact registry** (§5).1605. **Network** (DNS, proxy/egress, `conn.log`): indispensable for investigating, mediocre for161 detecting on its own.162163### 3.3 ATT&CK coverage with honesty164165- Coverage is scored by **visibility quality × detection quality**, not by166 rule count. Minimum scale per technique: `no telemetry` → `telemetry with no rule` →167 `unvalidated rule` → `rule validated against a real execution` → `rule validated and with168 a response procedure`. Only the last two count as coverage.169- **A technique covered by a single variant is not covered.** ATT&CK enumerates170 behaviours, not commands: if the rule only matches the binary the atomic test used, you have171 covered the tool, not the technique.172- Prioritise by **relevant threat** (adversary profile, sector, real surface), not by173 filling the matrix. A uniformly green matrix is a sign the colour has been optimised.174- **Generate the Navigator layers from the repo in CI.** A hand-edited coverage map175 lies as soon as the next PR is merged.176- When jumping ATT&CK versions, **use the official crosswalk** (the split of Defense177 Evasion in v19 invalidates previous mappings) and treat the migration as a change with a PR and178 review, not as a `sed`.179180### 3.4 Anatomy of a rule that can be governed181182Mandatory metadata, no exceptions: a stable `id` (UUID, never reused), an actionable183`title`, `status` (`experimental` → `test` → `stable` → `deprecated`), a `description` with184the hypothesis, `author`/`owner` (a team, not a person), `date`/`modified`, a justified `level`,185ATT&CK `tags` with the framework version, `falsepositives` **populated with real observed186cases**, `references` and **a link to the runbook**.187188- **Severity is earned, not declared.** A rule is born in `experimental` with informational189 severity; it goes up only with data: N days in production, a measured FP rate, and190 adversarial validation passed. Raising the severity "because the technique is serious" is the direct route to191 alert fatigue.192- **Actionable context in the event itself**: who (a resolved identity, not a GUID), which193 asset (with owner and criticality), what happened (relevant raw command/fields), what is expected194 of the analyst. An alert that forces you to open four consoles to know whether it matters has already195 failed.196- **Enrichment in the pipeline, not in the analyst's head**: asset inventory,197 criticality, owner, geolocation, reputation, and **whether that activity was198 authorised** (change window, deconflicted red team exercise).199- **Deduplication and grouping**: the unit of work is the **incident**, not the alert. It is200 grouped by the entities that define *one and the same case* (identity, host, session, campaign) with201 a time window. **Sigma v2 correlations** (`event_count`, `value_count`,202 `temporal`) are the portable mechanism for expressing chains and thresholds without tying yourself to the engine.203- **Portability with a declared limit**: Sigma is the authoring format; the backend translates.204 When the semantics do not survive translation (complex correlation, joins, engine-specific205 functions, sequence analysis), **write it in the native language and declare206 the exception in the rule** — do not force a Sigma that translates into something other than what it says.207- **Freshness of pySigma backends: verify it, do not trust the label.** The208 official directory marks abandoned backends as *stable* (`sentinelone` unpublished209 since 2023, `ibm-qradar-aql` and `insightidr` equally behind). Before betting a210 platform on a backend, look at the date of the last release, not the badge.211212### 3.5 Mandatory runbook: a rule with no action is not deployed213214Every alert carries a runbook versioned **in the same repo and in the same PR**, with: what the215firing means, **how it is verified in 5 minutes** (concrete queries, not "investigate"), what is216known benign, what immediate action is appropriate (contain, revoke session, isolate,217escalate), to whom it is escalated and at what severity, and which data to preserve before touching anything218(preservation belongs to `incident-response-forensics-standards`; the link is mandatory).219220**If you cannot write the runbook, you do not know what the rule detects: it is not merged.**221222### 3.6 Adversarial validation and regression detection223224- **No rule leaves `experimental` without having fired against a real execution of the225 technique.** The attack generator belongs to `offensive-security-standards`; the evidence of226 the firing is my deliverable.227- **Purple team ≠ red team**: the value is not "they got in", it is the **executed228 vs. detected vs. alerted vs. responded** matrix by technique. Every undetected execution opens229 a backlog item with an owner and a date.230- **Regressions**: a rule that used to work stops working on its own when the agent changes,231 or the vendor's schema, the OS version or a binary's path. **Scheduled and automatic**232 execution of a subset of atomics against a controlled environment, with an alert if the expected233 detection does not appear. A real and recent example of why this is not theoretical: Defender's234 *advanced hunting* schema renamed `AADSignInEventsBeta` to235 `EntraIdSignInEvents` and **removed the legacy tables on 9 Dec 2025**, and on 25 Feb 2026 the236 booleans went from `1`/`0` to `True`/`False` — each of those changes breaks rules237 silently.238- Every adversarial execution is **deconflicted with the SOC** beforehand: an exercise that consumes the239 real incident process is a process failure, not a test passed.240241### 3.7 Identity and cloud: where the detection that matters lives today242243The perimeter is identity. The minimum sources, with **exact names** (verify §8):244245- **Entra ID**: `SigninLogs` (interactive), `AADNonInteractiveUserSignInLogs`,246 `AADServicePrincipalSignInLogs`, `AADManagedIdentitySignInLogs`, `AuditLogs` and247 **`MicrosoftGraphActivityLogs`** (requires P1/P2). In 2026 **Agent logs** were added for248 *Entra Agent ID*, and `AADNonInteractiveUserSignInLogs` now carries an `Agent` column: AI249 agent identities are a new surface and must be ingested from day one.250- **AWS CloudTrail**: there are now **four** categories, not two — *Management*, *Data*, ***Network251 activity*** (`eventCategory = NetworkActivity`; the only valid `errorCode` is252 `VpceAccessDenied`, useful for seeing credentials outside your organisation using your VPC253 endpoints) and *Insights*. **Only management is on by default**: data and network events254 are enabled explicitly and budgeted for.255- **GCP**: `cloudaudit.googleapis.com%2Factivity` (Admin Activity, always),256 `%2Fdata_access` (**disabled by default** except for BigQuery), `%2Fsystem_event`,257 `%2Fpolicy`. For consent abuse in Workspace, the key log is the **OAuth Token258 Audit** (`applicationName=token`, `authorize`/`revoke` events, with `scope_data`,259 `client_id`, `app_name`); default retention **6 months** — if your investigation260 window is longer, export it.261- **GuardDuty Extended Threat Detection** is on by default and at no extra cost, with a rolling262 24 h window. **Verified operational trap: it ignores archived findings,263 including those archived by your own suppression rules** — aggressive tuning blinds you to264 attack sequence correlation.265266**Mandatory identity use cases** (defensible ATT&CK chain:267`T1566.002 → T1528 → T1550.001 → T1098.005 / T1098.001`; **there is no dedicated sub-technique for268device code phishing** — the one that mentions it explicitly is T1566.002, not T1528):269270- **Device code phishing**: the decisive field in Entra is `AuthenticationProtocol == "deviceCode"`.271 It is the technique of **Storm-2372** (Microsoft, Feb 2025; **reclassified on 31 Jul 2026 as a272 sub-cluster of Midnight Blizzard/SVR**), which pivoted to the client ID273 `29d9ed98-a469-4536-ade2-f981bc1d605e` (*Microsoft Authentication Broker*) to obtain a274 **PRT** and register a device. Preventive control: the Conditional Access **"Authentication flows"**275 condition. Planning notice: Microsoft is rolling out a **managed policy276 "Block device code flow"** which it creates in *report-only* and **enables on its own after a minimum of 45277 days**, warning 28 days in advance; audit it by filtering `AuditLogs` on the initiator278 `Microsoft Managed Policy Manager`. Critical caveat: **a CA policy targeting users279 does not cover service principals**.280- **Device registration and new credential**: adding or changing credentials on281 *service principals* and *app registrations*. It is exactly the pattern of the282 **Commvault/Metallic** case (CISA, May 2025, **CVE-2025-3928** in KEV): theft of *client secrets*283 from a cross-tenant app.284- **Abuse of a third-party integration's OAuth token**: the **Salesloft Drift / UNC6395** case285 (Aug 2025) exfiltrated Salesforce data with legitimate OAuth tokens, searching with mass286 SOQL for strings such as `AKIA`, `Snowflake`, `password`, `secret`, and **deleting the records of287 the query jobs**. Signals: Salesforce Event Monitoring `UniqueQuery` events, UA288 `Salesforce-Multi-Org-Fetcher/1.0`. Design lesson: **a third-party app's consent289 is a permanent key with no MFA** — inventory and watch consents the way you290 watch administrators.291- **Impossible travel and session anomaly**: useful, but **only as enrichment or a low-severity292 signal**. In a workforce with a corporate VPN and mobiles it generates massive FPs; never293 as a paging alert on its own.294- **Graph↔sign-in correlation** (verified canonical pattern):295 `MicrosoftGraphActivityLogs | join ... on $left.SignInActivityId == $right.UniqueTokenIdentifier`296 — it is what turns "a token did things" into "this session, of this user, from this297 origin, did these things".298299### 3.8 Threat intelligence: pyramid of pain, not shopping list300301- **IoCs expire; TTPs do not.** Hashes and IPs are rotated in hours and their value is almost purely302 retrospective; tools and TTPs cost the adversary and survive months or years. The303 engineering effort goes to the **top** of the pyramid; IoCs are automated and forgotten.304- **Every IoC comes in with an expiry and a provenance.** A feed with no expiry date is a305 source of FPs that grows on its own. Indiscriminate free feeds generate more triage than they306 save: they are measured like any rule (§6) and cut if they do not pay off.307- **Systematic retro-hunt**: every new IoC is searched **backwards** in the cold retention, not308 only forwards. That is where the IoC does earn its keep.309- The ultimate goal of TI is not the list: it is **prioritising the detection backlog** by310 relevant adversary.311312## 4. Quality gates (they break the build)313314In increasing order of cost. The first four are blocking on every detection PR.3153161. **Lint and schema**: the rule parses, validates against the format's schema (Sigma spec 2.1,317 `detection-rules` TOML, the engine's schema) and **compiles with the target backend**318 (`sigma convert`). A rule that does not translate does not exist.3192. **Complete metadata**: a unique, non-reused `id`, owner, `status`, ATT&CK tags320 **existing in the declared version** of the framework, non-empty `falsepositives`.3213. **Runbook present and linked.** No runbook, no merge (§3.5). Non-negotiable.3224. **Unit test with positives and negatives**: at least one event that **must** fire and323 several that **must not** (the rule's own known FPs). Valid engines:324 `detection_rules test` (Elastic), `panther_analysis_tool test`, Hayabusa/Chainsaw over325 sample EVTX, or the backend's runner. **A rule with no negative test is not tested:326 it is declared.**3275. **Schema non-regression test**: the test events are validated against the current328 schema (ECS/OCSF/vendor table), so that a field rename breaks the build329 and not the detection in silence (§3.6).3306. **Coverage**: the Navigator layer is **regenerated in CI** and the diff is reviewed. Adding a331 rule that does not move the declared coverage is a sign of a duplicate.3327. **Adversarial validation** before promoting to `stable` or raising severity: a real333 atomic execution and evidence of the firing attached to the PR.3348. **Deployment by pipeline** with versioning and rollback: `Sentinel repositories` (GA335 Mar 2026), `elastic/detection-rules` with a *version lock* per stack branch, or the equivalent336 mechanism. **Manual deployment in the console is forbidden.**337338**Measurable promotion rule** (adjust the numbers, but fix some): `experimental` → `test`339after compiling and passing tests; `test` → `stable` after ≥ 2 weeks in production with a **measured340FP rate < 20 %** and adversarial validation passed. **It is measured before raising the severity,341never after.**342343## 5. Security of the detection programme itself344345- **The supply chain of security tooling is a first-order target, and 2026346 proved it.** The compromise of **Trivy** by the actor *TeamPCP* (**CVE-2026-33634**, CVSS347 9.4, in **CISA KEV** since 26 Mar 2026) went as far as **force-pushing 76 of 77 tags** of348 `trivy-action` and all 7 of `setup-trivy`, with a payload that **read the memory of the349 `Runner.Worker` process to extract masked secrets**; the same campaign hijacked the 35 tags350 of `Checkmarx/kics-github-action`. Analogous precedent: `tj-actions/changed-files`351 (**CVE-2025-30066**). Operational consequences, without exception:352 - **Pin by commit SHA** every GitHub Action in the detection pipeline; **pin by353 digest** every image; verify signatures and checksums of the detection binaries.354 - **Immutable releases** enabled in your own content repos.355 - **Do not confuse provenance with goodness**: the *CanisterWorm* variant went as far as **generating356 valid SLSA Build L3 attestations via Sigstore for malicious packages**. The signature357 proves **who** built it, not **what** it does.358 - The detail of pipeline hardening belongs to `cicd-standards`; here it is an entry359 requirement.360- **Verified Aug 2026: none of the recommended detection tools (Sigma,361 YARA/YARA-X, Atomic Red Team, Caldera, Falco, Suricata, Zeek, Wazuh, Elastic) appears as a362 victim of a repository compromise in 2025-2026.** The vector, however, applies to them363 just the same: do not treat it as ruled out, but as not having happened yet.364- **The SIEM and its rules are sensitive data.** The ruleset reveals exactly what you see and what365 you do not: access control on the detection repo, review of who can disable or modify366 rules, and **an alert on the modification or silencing of the content itself**. Disabling367 a rule is a security event.368- **Never embed secrets in rules or in queries** (API tokens, enrichment369 credentials, webhook keys). They go to the secrets manager370 (`secrets-management-standards`).371- **PII and minimisation**: security logs contain personal data by definition.372 Legal basis, retention by purpose and access control belong to373 `privacy-engineering-standards`; the requirement here is that **the rule and its enrichment do not374 widen the set of personal data** beyond what is needed to decide.375- **Integrity and non-repudiation** of the evidence: append-only logs, a synchronised and376 verified clock (clock drift destroys any temporal correlation and any377 forensic timeline) and control over who can purge.378379## 6. Operability, cost and programme metrics380381**Metrics that are published (and what corrupts them)**382383| Metric | What it measures | How it gets corrupted |384|---|---|---|385| **MTTD** | Time from the adversary's activity to detection | Measuring from ingestion, not from the event; excluding what was never detected (survivorship bias). It is only honest with purple team, where you know the real `t0` |386| **Coverage per source** | % of assets of each class reporting the expected source | Counting configured sources instead of sources **that are sending events now** |387| **FP rate per rule** | Firings closed as benign / total firings | Not recording the analyst's disposition; then there is no data and tuning is opinion |388| **Alerts per analyst per hour** | Real triage load | Counting alerts instead of **grouped incidents**. An alarm threshold for the programme, not for the shift |389| **Orphaned rules** | No owner, no firings, no runbook, no review within its window | None: it is the metric hardest to dress up and the one that best predicts the programme's collapse |390| **% of adversarially validated rules** | Real coverage | Counting written rules |391392- **Alert fatigue is the domain's modal failure.** If the analyst ignores the queue,393 detection does not exist even if the rules are perfect. Work priority when the queue394 overflows: **reduce noise before adding coverage**, always.395- **Tuning ≠ disabling.** The correct order is: (1) fix the data or the enrichment,396 (2) narrow the rule's condition, (3) a **narrow, documented exception with an397 expiry**, (4) downgrade to informational, (5) retire. A broad and permanent suppression is398 a detection hole shaped like tuning — and on AWS it also blinds GuardDuty's399 correlation (§3.7).400- **Cost**: the SIEM is paid for by ingestion, by retention or by query compute depending on the401 product — **find out your billing unit before designing**, because it decides what to optimise.402 Levers in order: do not ingest what does not enable detection; filter and aggregate at the edge;403 tiered retention to cheap storage; efficient scheduled queries. Ingesting "just in404 case" is the most expensive way of detecting nothing.405- **Rule performance**: scheduled queries with a coherent window and frequency406 (window ≥ frequency + ingestion latency, or you lose events at the edge), no joins407 over the whole history, no catastrophic regexes. A rule that does not finish within its408 interval does not detect: it skips executions.409- **Platform change with a hard date**: if you use **Microsoft Sentinel**, the product is already410 GA in the Defender portal (Jan 2026) and **stops being supported in the Azure portal411 after 31 Mar 2027**. It is a migration project with a date, not a UI change.412413## 7. Sustainability and prohibitions414415- **Cadence**: review ATT&CK and remap on every major framework release (~half-yearly;416 v18 and v19 brought structural breaks); review the entire ruleset **quarterly** with417 explicit pruning; update `pySigma`/backends and the community ruleset monthly;418 review sources and their health continuously.419- **Every exception and every suppression carries an expiry and an owner.** Without a date, they become420 permanent holes nobody remembers opening.421- Third-party content is **calibrated on adoption**, not "some day": it enters the repo, passes422 the gates of §4 and is assigned an owner, or it does not enter.423- When a project changes governance (Caldera → Apache Incubator; Panther → Databricks),424 **review it as a platform risk**, not as a press release: licence, release cadence425 and who responds to a CVE.426427**FORBIDDEN**428- ❌ Deploying a rule **without a unit test** (positive **and** negative).429- ❌ Deploying a rule **without a runbook** with a concrete action.430- ❌ **Raising the severity without having measured** the FP rate in production.431- ❌ Creating or editing content **directly in the SIEM console**, with no repo and no PR.432- ❌ Enabling the product's default rule pack **without calibrating** — the domain's central433 antipattern: a SIEM full of rules nobody has touched is not coverage, it is a434 noise queue that guarantees the good alert goes unnoticed.435- ❌ Presenting a **count of rules or techniques** as coverage without scoring visibility and436 validation.437- ❌ Rules with no owner, no stable `id`, no ATT&CK version tags or with empty438 `falsepositives`.439- ❌ Writing rules for a source **not confirmed in ingestion**, or leaving the **absence of events**440 from a source without an alert.441- ❌ Broad, permanent or ownerless suppressions; silencing a rule instead of fixing it.442- ❌ Alerting on IoCs with no expiry or provenance, or building the programme on IoCs instead443 of on TTPs.444- ❌ Impossible travel (or other pure anomaly heuristics) as a standalone paging alert.445- ❌ Consuming GitHub Actions or images in the detection pipeline **by mutable tag** instead of446 by SHA/digest, or treating an SLSA attestation as proof that the artifact is benign.447- ❌ Running adversarial validation **without prior deconfliction** with the SOC.448- ❌ Secrets or credentials embedded in rules, queries or enrichment pipelines.449- ❌ Starting anything new on **Matano** or **DetectionLab** (unmaintained), or on classic450 YARA when YARA-X is available.451- ❌ Running Wazuh or Caldera on versions earlier than the minimums of §2 (critical CVEs, one452 of them mass-exploited).453- ❌ Forcing into Sigma a logic the backend does not support and trusting the454 result.455456## 8. Mandatory web verification457458Before pinning any version, licence, project status or table/field name,459**look it up — do not recall it**. Data verified Aug 2026 (it expires fast): Sigma spec460**v2.1.0**, pySigma **1.5.0**, sigma-cli **3.1.0**, SigmaHQ rules **r2026-07-01**; ATT&CK461**v19.1** (Apr 2026) and Navigator **5.3.2**; ATT&CK Workbench **4.10.0**; OCSF **1.8.0** (LF462since Nov 2024); ECS **9.4.0**; Elasticsearch **9.3.0** / Elastic Security **9.4.4**; Wazuh463**4.14.7**; Suricata **8.0.6** and **7.0.17**; Zeek **8.0.9**; Falco **0.44.1** (CNCF graduated464Feb 2024); YARA-X **1.19.0** and YARA **4.5.8**; Splunk ESCU **6.3.0**; `panther-analysis`465**3.112.0**; Chainsaw **2.16.2**; Hayabusa **3.10.0**; DeTT&CT **2.2.0**.4664671. **The current ATT&CK version and its changelog**, including the *crosswalk* for the split of468 Defense Evasion (v19) and the Detection Strategies/Analytics model (v18) — **before**469 remapping anything.4702. **The real freshness of the pySigma backend** you are going to use: the date of the last release in the471 `pySigma-plugin-directory`, not the `stable` badge.4723. **Governance and health** of Caldera (Apache Incubator since May 2026, **current version not473 verified here**) and of Panther (acquisition by Databricks announced Jun 2026, **future474 of the OSS not declared**).4754. **Exact names of the vendor's tables, fields and logs** before writing the query:476 the Defender/Entra, CloudTrail and Cloud Audit Logs schemas changed in 2025-2026 and477 keep changing.4785. **Hard platform dates**: end of support for Sentinel in the Azure portal479 (31 Mar 2027, verify), status of the content migrations under way.4806. **Advisories and CVEs** for your SIEM and your agents (Wazuh and Elastic have had recent481 criticals) and for the tooling you run in CI.4827. **Supply chain compromises** of any new action, image or binary in the483 detection pipeline before adopting it (§5).484485**Declared gaps — not verified in this document, verify them yourself before using them**:486- **The current version of Apache Caldera (incubating)**.487- **The version of Splunk Enterprise Security** and product news after the acquisition by488 Cisco (the official documentation was not accessible during verification).489- **Formal EOL dates for the Suricata 7.0 and 8.0 branches**, and whether a 9.0 release exists or is490 planned (the upgrade guide mentions it; no release has been published).491- **The status of OCSF support in Elastic Security and in Splunk** in 2026.492- **Whether Red Canary (maintainer of Atomic Red Team) is today part of Zscaler** — a secondary493 source, unconfirmed against a primary release.494- **GA dates** for the Conditional Access "Authentication flows" condition, for CloudTrail's495 *network activity events* and for GuardDuty Extended Threat Detection.496- **Secondary ATT&CK IDs** cited in identity literature (T1621, T1539, T1550.004,497 T1078.004, T1098.002/.003, T1606.002): not individually verified against the site.498- **Whether the recommended detection projects consumed** `trivy-action` or499 `tj-actions/changed-files` during their exposure windows: not ruled out, just not500 found.501502If the web contradicts this document, **the web wins** — flag the discrepancy.