Detection engineering standards
Criteria verified as of August 2026. Re-verify on the web before committing to anything (§8).
1. Scope and triggers
Applies to designing, writing, testing, deploying, measuring and retiring detection content:
detection backlog and hypotheses, data model and source onboarding, schema
normalisation (OCSF, ECS), rule authoring in any format (Sigma, YARA/YARA-X, KQL, SPL,
EQL, YARA-L, Suricata, Zeek, Wazuh, Falco), the detection-as-code pipeline (repo, CI,
tests, deployment, versioning), coverage map against MITRE ATT&CK, enrichment and
deduplication, thresholds and severity, false positive management, per-alert runbook,
adversarial validation of the detection, use of threat intelligence, and the programme's
metrics (MTTD, coverage per source, FP per rule, alerts per analyst per hour).
Triggers: Sigma *.yml with detection:/logsource:, sigma-cli/sigmac, pySigma,
*.yar/*.yara, yr (YARA-X), rules/**/*.toml from elastic/detection-rules,
detection_rules test, contentctl/contentctl-ng, panther_analysis_tool, suricata.yaml
and *.rules, *.zeek scripts, Wazuh local_rules.xml/decoders, falco_rules.local.yaml,
ATT&CK Navigator .json layers, chainsaw, hayabusa, atomic-red-team, "detection
rule", "SIEM use case", "false positive", "tuning", "ATT&CK coverage", "MTTD".
Guiding principle: detection is an engineering product with a lifecycle, not a
catalogue of bought alerts. It is governed by three invariants: (1) without the right source there is
no detection — the data model precedes the rule; (2) every rule is tested with the
real attack it claims to detect before it is believed; (3) an alert with no possible action is not
an alert, it is noise with an owner. Uncomfortable corollary: coverage ≠ detection. "We cover
300 techniques" almost always means "we have 300 rules nobody has run against an
attack".
Not applicable: see observability-standards (telemetry for diagnosing: OpenTelemetry,
Prometheus, Loki, cardinality, cost, Alertmanager — the boundary is the purpose, not the
tool: the same log feeds both, there to explain an outage, here to discover
an adversary; the pipeline, retention and integrity of the telemetry are theirs, the
analytical content is mine), sre-practice-standards (on-call, SLOs, error budget, burn rate,
postmortems), incident-management-standards (governance of the incident my alert
triggers: declaration, severity, IC, communication — my deliverable ends exactly at the
handoff, and its retro sends work back to me), incident-response-forensics-standards (technical
response and investigation: containment, imaging, timeline, eradication — a
bidirectional boundary: there they investigate what I detect, and every investigation must come back
turned into a new rule; the use of YARA for forensic hunting and evidence triage is
theirs, the authoring and governance of the rule is mine), offensive-security-standards (purple
team, adversary emulation, Atomic Red Team and Caldera as generators of the authorised
attack and its deconfliction with the SOC — they generate the activity, I write and validate
the detection), container-runtime-security-standards (which runtime signal matters —
unexpected shell, writes to binaries, runtime socket, drift— and the deployment of the
eBPF agent: the Falco rule is born there and is governed here, with my lifecycle, my tests
and my coverage), linux-hardening-standards and selinux-standards (CIS baseline, auditd and
SELinux AVCs as a source: configuring the audit ruleset is theirs, turning
those events into detection is mine), windows-server-ad-standards (which AD event matters and
why), networking-standards (segmentation, NetFlow/IPFIX and the network Suricata/Zeek
observe), appsec-standards (A09 Security Logging and Alerting Failures: which events
the application must emit — authn, denied authz, privilege change; I consume that
emission, I do not design it), vulnerability-management-standards (CVE, CVSS/EPSS/KEV and patching
SLAs), privacy-engineering-standards (PII inside security logs, minimisation
and retention limits by purpose), secrets-management-standards (custody and rotation of
secrets — shared boundary: a leaked secret or a secret used from an anomalous origin is a
detection case of mine, fed by their manager and scanning telemetry), grc-compliance-standards (the
regulatory control requiring monitoring and its evidence), identity-access-management-standards (IdP design,
conditional access policies and OAuth flows — I detect their abuse, I do not configure them),
threat-intelligence-standards (bidirectional boundary: the hypothesis, the indicator and its
expiry are theirs, the rule is mine — and the return of sightings is theirs: an indicator that
expires for lack of sightings is retired there, by their criteria, not here).
2. Default decisions
Data verified Aug 2026. Verify the latest version, licence and status on the web before
pinning it in a real project (§8): this ecosystem rotates in months and several projects
changed governance in 2025-2026.
| Area |
Default |
Justifiable alternative / vetoed |
| Portable authoring format |
Sigma — spec v2.1.0 (Sep 2025, formalises correlations and meta-rules), pySigma 1.5.0, sigma-cli 3.1.0, SigmaHQ/sigma rules r2026-07-01 |
Writing directly in the SIEM's language only when translation loses semantics (§3). Hard minimum: pySigma ≥ 1.3.0 — earlier versions executed code via template vars (today behind --enable-template-vars and a sandboxed Jinja2) |
| Files and memory |
YARA-X 1.19.0 |
Classic YARA 4.5.8 only for third-party engine compatibility: VirusTotal declared it in maintenance mode when publishing YARA-X 1.0.0 (Jun 2025) and new modules (macho, lnk) only reach YARA-X. Migrate while validating the ruleset (≈99 % compatible at rule level) |
| Threat reference framework |
MITRE ATT&CK v19.1 (Apr 2026) + Navigator 5.3.2 |
None. It is the lingua franca. Watch two breaks: v18.0 replaced Detections with Detection Strategies (DETxxxx) + Analytics (ANxxxx) and deprecated the legacy Data Sources; v19 split Defense Evasion into Stealth (TA0005) and Defense Impairment (TA0112) — use the official crosswalk, do not renumber by hand |
| Coverage measurement |
DeTT&CT 2.2.0 over Navigator layers, scoring data source quality, not just rule presence |
Your own spreadsheet (it desynchronises on its own). Vetoed: presenting "techniques with a rule" as coverage without scoring visibility |
| Adversarial validation |
Atomic Red Team (MIT, rolling model with no tags) for atomic tests per technique |
Apache Caldera for chains and campaign emulation — it changed governance: MITRE contributed it to the Apache Incubator on 20 May 2026, canonical repo apache/caldera. Hard minimum ≥ 5.1.0 because of CVE-2025-27364 (CVSS 10.0, preauth RCE in agent compilation) |
| Normalisation schema |
The one of the engine you consume, declared explicitly: ECS 9.4.0 in the Elastic/Sigma world; OCSF 1.8.0 (Linux Foundation since Nov 2024) in a data lake and for multi-vendor interoperability |
There is no single winner today. Frequent correction: ECS was not donated to OCSF, but to OpenTelemetry (Apr 2023), and that convergence is still incomplete. Pick one per platform and write it down; translate at the edge, not in the rule |
| Low-cost self-hosted SIEM |
Wazuh 4.14.7 (GPLv2) |
Hard minimum 4.14.4+. Wazuh has a serious history: CVE-2025-24016 (CVSS 9.9, deserialisation in DAPI, fix 4.9.1) was actively exploited by Mirai botnets. It is a server exposed to agents: treat it as Tier 0 |
| Commercial SIEM |
Whichever you already have; portability comes from Sigma, not the product |
Elastic Security 9.4.x (triple licence: AGPLv3 / SSPL / Elastic License — verify which applies to your deployment); Microsoft Sentinel; Google SecOps (formerly Chronicle) with YARA-L 2.0; Splunk ES |
| Network — IDS/IPS |
Suricata (8.0.6 and 7.0.17 in parallel, GPLv2) for signatures and protocol detection |
Zeek 8.0.9 (BSD) for network metadata and context: they do not compete, they complement each other. Zeek gives you the conn.log/ssl.log that makes the Suricata alert investigable |
| Container runtime |
Falco 0.44.1 (Apache-2.0, CNCF graduated since Feb 2024) |
The choice of agent and the what to watch belongs to container-runtime-security-standards; here the content lifecycle |
| Offline EVTX triage |
Hayabusa 3.10.0 (AGPL-3.0, ATT&CK v19) or Chainsaw 2.16.2 (GPL-3.0) — both run Sigma over evidence |
Manual EVTX parsing. Also useful as a test engine for Windows Sigma rules in CI |
| Third-party content as a base |
SigmaHQ (rules), elastic/detection-rules, splunk/security_content (ESCU 6.3.0, Apache-2.0), panther-analysis 3.112.0 (Apache-2.0), Nextron signature-base (DRL 1.1 licence — check before using in a commercial product) |
Consuming the whole package without calibrating (§7). Vetoed: closed vendor rules whose logic you can neither read nor version |
Projects not to choose in 2026 (verified): Matano — no commits since Jan 2025;
DetectionLab — declared dead by its author since 2023. Panther: Databricks announced an
acquisition agreement on 16 Jun 2026 and the release says nothing about the future of the OSS —
panther-analysis is still Apache-2.0 and active, but it is a bet with governance risk.
Worth mentioning in case it applies to the European public sector: OpenTIDE (European Commission, EUPL 1.2,
org OpenTideHQ) is today the most complete formal open-source detection-as-code framework.
3. Structure and conventions
3.1 The lifecycle (a rule is a software artifact, not a ticket)
idea/hypothesis → data source → rule → test → deployment by CI → measurement → tuning → retirement
- Idea/hypothesis: every rule is born from a written hypothesis (
"an attacker with valid credentials will register a new device in Entra after device code phishing"), with an
ATT&CK technique, a reference actor or campaign, and expected value. Without a hypothesis there is no
criterion for knowing whether the rule works or for retiring it.
- Source before rule (§3.2): if the event does not arrive, the rule is theatre.
- Rule: in the repo, with complete metadata (§3.4), reviewed in a PR by somebody other
than the author.
- Test: mandatory unit test (§4), plus adversarial validation before raising the severity.
- Deployment by CI: never by hand in the SIEM console. The console is read-only for
content; what is edited there is lost at the next sync or produces invisible
drift.
- Measurement: FP rate, firings, actions taken. A rule with no telemetry about itself
cannot be governed.
- Retirement: a rule that has fired nothing useful in its review window is deleted or
downgraded to informational, in the same PR that establishes this. The cost of a dead rule is
not zero: it consumes compute, dirties the coverage and wears down the analyst who triages it.
Reference repo structure (adapt it, but have one):
detections/
rules/<platform>/<tactic>/<id>.yml # Sigma or the engine's native format
pipelines/<platform>.yml # pySigma processing pipelines
tests/<id>/{positive,negative}.jsonl # events that MUST and MUST NOT fire
runbooks/<id>.md # mandatory (§3.5)
coverage/navigator-*.json # generated layers, not hand-edited
deprecated/ # with reason and date, not silent deletion
3.2 Data model and telemetry: the part almost nobody does
- Onboarding a source is a project, not a checkbox. For every source:
which concrete events (by ID/category, not "the AD logs"), how they are transported,
how they are normalised, how long they are retained, what they cost and which detections they enable.
If it enables none, it is not ingested.
- Cost and retention are first-class design constraints, exactly as in
observability-standards. Healthy pattern: short hot retention (interactive search) +
long, cheap cold retention (investigation and compliance). Practical rule: the cold
window must cover the dwell time you assume, not the one you would like.
- Normalise at the edge, not in the rule. A rule that slices strings to reconstruct a
field the pipeline should have normalised is debt paid on every new source.
- Measure source health as if it were an SLI: ingestion latency, volume per hour
with an expected band, and alert on the absence of events — a source's silence is
indistinguishable from "nothing happened", and it is exactly what an attacker who
disables the agent produces. Detection of missing detection is the first rule you write.
- Integrity: security logs are kept separate from operational telemetry (different access
control, retention and immutability). The SIEM is the record an attacker
will want to erase.
Source priority today (by value/cost ratio, not by tradition):
- Identity and the cloud control plane — it is the main surface (§3.7).
- Endpoint/EDR (process, command line, parent/child, module loading).
- Authentication and directory (AD, Entra, IdP).
- CI/CD and the artifact registry (§5).
- Network (DNS, proxy/egress,
conn.log): indispensable for investigating, mediocre for
detecting on its own.
3.3 ATT&CK coverage with honesty
- Coverage is scored by visibility quality × detection quality, not by
rule count. Minimum scale per technique:
no telemetry → telemetry with no rule →
unvalidated rule → rule validated against a real execution → rule validated and with a response procedure. Only the last two count as coverage.
- A technique covered by a single variant is not covered. ATT&CK enumerates
behaviours, not commands: if the rule only matches the binary the atomic test used, you have
covered the tool, not the technique.
- Prioritise by relevant threat (adversary profile, sector, real surface), not by
filling the matrix. A uniformly green matrix is a sign the colour has been optimised.
- Generate the Navigator layers from the repo in CI. A hand-edited coverage map
lies as soon as the next PR is merged.
- When jumping ATT&CK versions, use the official crosswalk (the split of Defense
Evasion in v19 invalidates previous mappings) and treat the migration as a change with a PR and
review, not as a
sed.
3.4 Anatomy of a rule that can be governed
Mandatory metadata, no exceptions: a stable id (UUID, never reused), an actionable
title, status (experimental → test → stable → deprecated), a description with
the hypothesis, author/owner (a team, not a person), date/modified, a justified level,
ATT&CK tags with the framework version, falsepositives populated with real observed
cases, references and a link to the runbook.
- Severity is earned, not declared. A rule is born in
experimental with informational
severity; it goes up only with data: N days in production, a measured FP rate, and
adversarial validation passed. Raising the severity "because the technique is serious" is the direct route to
alert fatigue.
- Actionable context in the event itself: who (a resolved identity, not a GUID), which
asset (with owner and criticality), what happened (relevant raw command/fields), what is expected
of the analyst. An alert that forces you to open four consoles to know whether it matters has already
failed.
- Enrichment in the pipeline, not in the analyst's head: asset inventory,
criticality, owner, geolocation, reputation, and whether that activity was
authorised (change window, deconflicted red team exercise).
- Deduplication and grouping: the unit of work is the incident, not the alert. It is
grouped by the entities that define one and the same case (identity, host, session, campaign) with
a time window. Sigma v2 correlations (
event_count, value_count,
temporal) are the portable mechanism for expressing chains and thresholds without tying yourself to the engine.
- Portability with a declared limit: Sigma is the authoring format; the backend translates.
When the semantics do not survive translation (complex correlation, joins, engine-specific
functions, sequence analysis), write it in the native language and declare
the exception in the rule — do not force a Sigma that translates into something other than what it says.
- Freshness of pySigma backends: verify it, do not trust the label. The
official directory marks abandoned backends as stable (
sentinelone unpublished
since 2023, ibm-qradar-aql and insightidr equally behind). Before betting a
platform on a backend, look at the date of the last release, not the badge.
3.5 Mandatory runbook: a rule with no action is not deployed
Every alert carries a runbook versioned in the same repo and in the same PR, with: what the
firing means, how it is verified in 5 minutes (concrete queries, not "investigate"), what is
known benign, what immediate action is appropriate (contain, revoke session, isolate,
escalate), to whom it is escalated and at what severity, and which data to preserve before touching anything
(preservation belongs to incident-response-forensics-standards; the link is mandatory).
If you cannot write the runbook, you do not know what the rule detects: it is not merged.
3.6 Adversarial validation and regression detection
- No rule leaves
experimental without having fired against a real execution of the
technique. The attack generator belongs to offensive-security-standards; the evidence of
the firing is my deliverable.
- Purple team ≠ red team: the value is not "they got in", it is the executed
vs. detected vs. alerted vs. responded matrix by technique. Every undetected execution opens
a backlog item with an owner and a date.
- Regressions: a rule that used to work stops working on its own when the agent changes,
or the vendor's schema, the OS version or a binary's path. Scheduled and automatic
execution of a subset of atomics against a controlled environment, with an alert if the expected
detection does not appear. A real and recent example of why this is not theoretical: Defender's
advanced hunting schema renamed
AADSignInEventsBeta to
EntraIdSignInEvents and removed the legacy tables on 9 Dec 2025, and on 25 Feb 2026 the
booleans went from 1/0 to True/False — each of those changes breaks rules
silently.
- Every adversarial execution is deconflicted with the SOC beforehand: an exercise that consumes the
real incident process is a process failure, not a test passed.
3.7 Identity and cloud: where the detection that matters lives today
The perimeter is identity. The minimum sources, with exact names (verify §8):
- Entra ID:
SigninLogs (interactive), AADNonInteractiveUserSignInLogs,
AADServicePrincipalSignInLogs, AADManagedIdentitySignInLogs, AuditLogs and
MicrosoftGraphActivityLogs (requires P1/P2). In 2026 Agent logs were added for
Entra Agent ID, and AADNonInteractiveUserSignInLogs now carries an Agent column: AI
agent identities are a new surface and must be ingested from day one.
- AWS CloudTrail: there are now four categories, not two — Management, Data, Network
activity (
eventCategory = NetworkActivity; the only valid errorCode is
VpceAccessDenied, useful for seeing credentials outside your organisation using your VPC
endpoints) and Insights. Only management is on by default: data and network events
are enabled explicitly and budgeted for.
- GCP:
cloudaudit.googleapis.com%2Factivity (Admin Activity, always),
%2Fdata_access (disabled by default except for BigQuery), %2Fsystem_event,
%2Fpolicy. For consent abuse in Workspace, the key log is the OAuth Token
Audit (applicationName=token, authorize/revoke events, with scope_data,
client_id, app_name); default retention 6 months — if your investigation
window is longer, export it.
- GuardDuty Extended Threat Detection is on by default and at no extra cost, with a rolling
24 h window. Verified operational trap: it ignores archived findings,
including those archived by your own suppression rules — aggressive tuning blinds you to
attack sequence correlation.
Mandatory identity use cases (defensible ATT&CK chain:
T1566.002 → T1528 → T1550.001 → T1098.005 / T1098.001; there is no dedicated sub-technique for
device code phishing — the one that mentions it explicitly is T1566.002, not T1528):
- Device code phishing: the decisive field in Entra is
AuthenticationProtocol == "deviceCode".
It is the technique of Storm-2372 (Microsoft, Feb 2025; reclassified on 31 Jul 2026 as a
sub-cluster of Midnight Blizzard/SVR), which pivoted to the client ID
29d9ed98-a469-4536-ade2-f981bc1d605e (Microsoft Authentication Broker) to obtain a
PRT and register a device. Preventive control: the Conditional Access "Authentication flows"
condition. Planning notice: Microsoft is rolling out a managed policy
"Block device code flow" which it creates in report-only and enables on its own after a minimum of 45
days, warning 28 days in advance; audit it by filtering AuditLogs on the initiator
Microsoft Managed Policy Manager. Critical caveat: a CA policy targeting users
does not cover service principals.
- Device registration and new credential: adding or changing credentials on
service principals and app registrations. It is exactly the pattern of the
Commvault/Metallic case (CISA, May 2025, CVE-2025-3928 in KEV): theft of client secrets
from a cross-tenant app.
- Abuse of a third-party integration's OAuth token: the Salesloft Drift / UNC6395 case
(Aug 2025) exfiltrated Salesforce data with legitimate OAuth tokens, searching with mass
SOQL for strings such as
AKIA, Snowflake, password, secret, and deleting the records of
the query jobs. Signals: Salesforce Event Monitoring UniqueQuery events, UA
Salesforce-Multi-Org-Fetcher/1.0. Design lesson: a third-party app's consent
is a permanent key with no MFA — inventory and watch consents the way you
watch administrators.
- Impossible travel and session anomaly: useful, but only as enrichment or a low-severity
signal. In a workforce with a corporate VPN and mobiles it generates massive FPs; never
as a paging alert on its own.
- Graph↔sign-in correlation (verified canonical pattern):
MicrosoftGraphActivityLogs | join ... on $left.SignInActivityId == $right.UniqueTokenIdentifier
— it is what turns "a token did things" into "this session, of this user, from this
origin, did these things".
3.8 Threat intelligence: pyramid of pain, not shopping list
- IoCs expire; TTPs do not. Hashes and IPs are rotated in hours and their value is almost purely
retrospective; tools and TTPs cost the adversary and survive months or years. The
engineering effort goes to the top of the pyramid; IoCs are automated and forgotten.
- Every IoC comes in with an expiry and a provenance. A feed with no expiry date is a
source of FPs that grows on its own. Indiscriminate free feeds generate more triage than they
save: they are measured like any rule (§6) and cut if they do not pay off.
- Systematic retro-hunt: every new IoC is searched backwards in the cold retention, not
only forwards. That is where the IoC does earn its keep.
- The ultimate goal of TI is not the list: it is prioritising the detection backlog by
relevant adversary.
4. Quality gates (they break the build)
In increasing order of cost. The first four are blocking on every detection PR.
- Lint and schema: the rule parses, validates against the format's schema (Sigma spec 2.1,
detection-rules TOML, the engine's schema) and compiles with the target backend
(sigma convert). A rule that does not translate does not exist.
- Complete metadata: a unique, non-reused
id, owner, status, ATT&CK tags
existing in the declared version of the framework, non-empty falsepositives.
- Runbook present and linked. No runbook, no merge (§3.5). Non-negotiable.
- Unit test with positives and negatives: at least one event that must fire and
several that must not (the rule's own known FPs). Valid engines:
detection_rules test (Elastic), panther_analysis_tool test, Hayabusa/Chainsaw over
sample EVTX, or the backend's runner. A rule with no negative test is not tested:
it is declared.
- Schema non-regression test: the test events are validated against the current
schema (ECS/OCSF/vendor table), so that a field rename breaks the build
and not the detection in silence (§3.6).
- Coverage: the Navigator layer is regenerated in CI and the diff is reviewed. Adding a
rule that does not move the declared coverage is a sign of a duplicate.
- Adversarial validation before promoting to
stable or raising severity: a real
atomic execution and evidence of the firing attached to the PR.
- Deployment by pipeline with versioning and rollback:
Sentinel repositories (GA
Mar 2026), elastic/detection-rules with a version lock per stack branch, or the equivalent
mechanism. Manual deployment in the console is forbidden.
Measurable promotion rule (adjust the numbers, but fix some): experimental → test
after compiling and passing tests; test → stable after ≥ 2 weeks in production with a measured
FP rate < 20 % and adversarial validation passed. It is measured before raising the severity,
never after.
5. Security of the detection programme itself
- The supply chain of security tooling is a first-order target, and 2026
proved it. The compromise of Trivy by the actor TeamPCP (CVE-2026-33634, CVSS
9.4, in CISA KEV since 26 Mar 2026) went as far as force-pushing 76 of 77 tags of
trivy-action and all 7 of setup-trivy, with a payload that read the memory of the
Runner.Worker process to extract masked secrets; the same campaign hijacked the 35 tags
of Checkmarx/kics-github-action. Analogous precedent: tj-actions/changed-files
(CVE-2025-30066). Operational consequences, without exception:
- Pin by commit SHA every GitHub Action in the detection pipeline; pin by
digest every image; verify signatures and checksums of the detection binaries.
- Immutable releases enabled in your own content repos.
- Do not confuse provenance with goodness: the CanisterWorm variant went as far as generating
valid SLSA Build L3 attestations via Sigstore for malicious packages. The signature
proves who built it, not what it does.
- The detail of pipeline hardening belongs to
cicd-standards; here it is an entry
requirement.
- Verified Aug 2026: none of the recommended detection tools (Sigma,
YARA/YARA-X, Atomic Red Team, Caldera, Falco, Suricata, Zeek, Wazuh, Elastic) appears as a
victim of a repository compromise in 2025-2026. The vector, however, applies to them
just the same: do not treat it as ruled out, but as not having happened yet.
- The SIEM and its rules are sensitive data. The ruleset reveals exactly what you see and what
you do not: access control on the detection repo, review of who can disable or modify
rules, and an alert on the modification or silencing of the content itself. Disabling
a rule is a security event.
- Never embed secrets in rules or in queries (API tokens, enrichment
credentials, webhook keys). They go to the secrets manager
(
secrets-management-standards).
- PII and minimisation: security logs contain personal data by definition.
Legal basis, retention by purpose and access control belong to
privacy-engineering-standards; the requirement here is that the rule and its enrichment do not
widen the set of personal data beyond what is needed to decide.
- Integrity and non-repudiation of the evidence: append-only logs, a synchronised and
verified clock (clock drift destroys any temporal correlation and any
forensic timeline) and control over who can purge.
6. Operability, cost and programme metrics
Metrics that are published (and what corrupts them)
| Metric |
What it measures |
How it gets corrupted |
| MTTD |
Time from the adversary's activity to detection |
Measuring from ingestion, not from the event; excluding what was never detected (survivorship bias). It is only honest with purple team, where you know the real t0 |
| Coverage per source |
% of assets of each class reporting the expected source |
Counting configured sources instead of sources that are sending events now |
| FP rate per rule |
Firings closed as benign / total firings |
Not recording the analyst's disposition; then there is no data and tuning is opinion |
| Alerts per analyst per hour |
Real triage load |
Counting alerts instead of grouped incidents. An alarm threshold for the programme, not for the shift |
| Orphaned rules |
No owner, no firings, no runbook, no review within its window |
None: it is the metric hardest to dress up and the one that best predicts the programme's collapse |
| % of adversarially validated rules |
Real coverage |
Counting written rules |
- Alert fatigue is the domain's modal failure. If the analyst ignores the queue,
detection does not exist even if the rules are perfect. Work priority when the queue
overflows: reduce noise before adding coverage, always.
- Tuning ≠ disabling. The correct order is: (1) fix the data or the enrichment,
(2) narrow the rule's condition, (3) a narrow, documented exception with an
expiry, (4) downgrade to informational, (5) retire. A broad and permanent suppression is
a detection hole shaped like tuning — and on AWS it also blinds GuardDuty's
correlation (§3.7).
- Cost: the SIEM is paid for by ingestion, by retention or by query compute depending on the
product — find out your billing unit before designing, because it decides what to optimise.
Levers in order: do not ingest what does not enable detection; filter and aggregate at the edge;
tiered retention to cheap storage; efficient scheduled queries. Ingesting "just in
case" is the most expensive way of detecting nothing.
- Rule performance: scheduled queries with a coherent window and frequency
(window ≥ frequency + ingestion latency, or you lose events at the edge), no joins
over the whole history, no catastrophic regexes. A rule that does not finish within its
interval does not detect: it skips executions.
- Platform change with a hard date: if you use Microsoft Sentinel, the product is already
GA in the Defender portal (Jan 2026) and stops being supported in the Azure portal
after 31 Mar 2027. It is a migration project with a date, not a UI change.
7. Sustainability and prohibitions
- Cadence: review ATT&CK and remap on every major framework release (~half-yearly;
v18 and v19 brought structural breaks); review the entire ruleset quarterly with
explicit pruning; update
pySigma/backends and the community ruleset monthly;
review sources and their health continuously.
- Every exception and every suppression carries an expiry and an owner. Without a date, they become
permanent holes nobody remembers opening.
- Third-party content is calibrated on adoption, not "some day": it enters the repo, passes
the gates of §4 and is assigned an owner, or it does not enter.
- When a project changes governance (Caldera → Apache Incubator; Panther → Databricks),
review it as a platform risk, not as a press release: licence, release cadence
and who responds to a CVE.
FORBIDDEN
- ❌ Deploying a rule without a unit test (positive and negative).
- ❌ Deploying a rule without a runbook with a concrete action.
- ❌ Raising the severity without having measured the FP rate in production.
- ❌ Creating or editing content directly in the SIEM console, with no repo and no PR.
- ❌ Enabling the product's default rule pack without calibrating — the domain's central
antipattern: a SIEM full of rules nobody has touched is not coverage, it is a
noise queue that guarantees the good alert goes unnoticed.
- ❌ Presenting a count of rules or techniques as coverage without scoring visibility and
validation.
- ❌ Rules with no owner, no stable
id, no ATT&CK version tags or with empty
falsepositives.
- ❌ Writing rules for a source not confirmed in ingestion, or leaving the absence of events
from a source without an alert.
- ❌ Broad, permanent or ownerless suppressions; silencing a rule instead of fixing it.
- ❌ Alerting on IoCs with no expiry or provenance, or building the programme on IoCs instead
of on TTPs.
- ❌ Impossible travel (or other pure anomaly heuristics) as a standalone paging alert.
- ❌ Consuming GitHub Actions or images in the detection pipeline by mutable tag instead of
by SHA/digest, or treating an SLSA attestation as proof that the artifact is benign.
- ❌ Running adversarial validation without prior deconfliction with the SOC.
- ❌ Secrets or credentials embedded in rules, queries or enrichment pipelines.
- ❌ Starting anything new on Matano or DetectionLab (unmaintained), or on classic
YARA when YARA-X is available.
- ❌ Running Wazuh or Caldera on versions earlier than the minimums of §2 (critical CVEs, one
of them mass-exploited).
- ❌ Forcing into Sigma a logic the backend does not support and trusting the
result.
8. Mandatory web verification
Before pinning any version, licence, project status or table/field name,
look it up — do not recall it. Data verified Aug 2026 (it expires fast): Sigma spec
v2.1.0, pySigma 1.5.0, sigma-cli 3.1.0, SigmaHQ rules r2026-07-01; ATT&CK
v19.1 (Apr 2026) and Navigator 5.3.2; ATT&CK Workbench 4.10.0; OCSF 1.8.0 (LF
since Nov 2024); ECS 9.4.0; Elasticsearch 9.3.0 / Elastic Security 9.4.4; Wazuh
4.14.7; Suricata 8.0.6 and 7.0.17; Zeek 8.0.9; Falco 0.44.1 (CNCF graduated
Feb 2024); YARA-X 1.19.0 and YARA 4.5.8; Splunk ESCU 6.3.0; panther-analysis
3.112.0; Chainsaw 2.16.2; Hayabusa 3.10.0; DeTT&CT 2.2.0.
- The current ATT&CK version and its changelog, including the crosswalk for the split of
Defense Evasion (v19) and the Detection Strategies/Analytics model (v18) — before
remapping anything.
- The real freshness of the pySigma backend you are going to use: the date of the last release in the
pySigma-plugin-directory, not the stable badge.
- Governance and health of Caldera (Apache Incubator since May 2026, current version not
verified here) and of Panther (acquisition by Databricks announced Jun 2026, future
of the OSS not declared).
- Exact names of the vendor's tables, fields and logs before writing the query:
the Defender/Entra, CloudTrail and Cloud Audit Logs schemas changed in 2025-2026 and
keep changing.
- Hard platform dates: end of support for Sentinel in the Azure portal
(31 Mar 2027, verify), status of the content migrations under way.
- Advisories and CVEs for your SIEM and your agents (Wazuh and Elastic have had recent
criticals) and for the tooling you run in CI.
- Supply chain compromises of any new action, image or binary in the
detection pipeline before adopting it (§5).
Declared gaps — not verified in this document, verify them yourself before using them:
- The current version of Apache Caldera (incubating).
- The version of Splunk Enterprise Security and product news after the acquisition by
Cisco (the official documentation was not accessible during verification).
- Formal EOL dates for the Suricata 7.0 and 8.0 branches, and whether a 9.0 release exists or is
planned (the upgrade guide mentions it; no release has been published).
- The status of OCSF support in Elastic Security and in Splunk in 2026.
- Whether Red Canary (maintainer of Atomic Red Team) is today part of Zscaler — a secondary
source, unconfirmed against a primary release.
- GA dates for the Conditional Access "Authentication flows" condition, for CloudTrail's
network activity events and for GuardDuty Extended Threat Detection.
- Secondary ATT&CK IDs cited in identity literature (T1621, T1539, T1550.004,
T1078.004, T1098.002/.003, T1606.002): not individually verified against the site.
- Whether the recommended detection projects consumed
trivy-action or
tj-actions/changed-files during their exposure windows: not ruled out, just not
found.
If the web contradicts this document, the web wins — flag the discrepancy.
1---2name: detection-engineering-standards3description: Security detection engineering — building and governing SIEM detections as code. Use when writing or tuning Sigma rules (sigma-cli, pySigma, sigma correlations), YARA or YARA-X rules, Elastic detection-rules TOML, KQL, SPL, EQL or YARA-L analytics, Suricata and Zeek signatures, Wazuh decoders and rules, ATT&CK Navigator coverage layers, DeTT&CT, Chainsaw or Hayabusa triage, alert runbooks and false-positive rates, detection unit tests in CI, OCSF or ECS log normalization, or onboarding a new log source into a SIEM.4---56# Detection engineering standards78Criteria verified as of **August 2026**. Re-verify on the web before committing to anything (§8).910## 1. Scope and triggers1112Applies to designing, writing, testing, deploying, measuring and **retiring** detection content:13detection backlog and hypotheses, data model and source onboarding, schema14normalisation (OCSF, ECS), rule authoring in any format (Sigma, YARA/YARA-X, KQL, SPL,15EQL, YARA-L, Suricata, Zeek, Wazuh, Falco), the detection-as-code pipeline (repo, CI,16tests, deployment, versioning), coverage map against MITRE ATT&CK, enrichment and17deduplication, thresholds and severity, false positive management, per-alert runbook,18adversarial validation of the detection, use of threat intelligence, and the programme's19metrics (MTTD, coverage per source, FP per rule, alerts per analyst per hour).2021Triggers: Sigma `*.yml` with `detection:`/`logsource:`, `sigma-cli`/`sigmac`, `pySigma`,22`*.yar`/`*.yara`, `yr` (YARA-X), `rules/**/*.toml` from `elastic/detection-rules`,23`detection_rules test`, `contentctl`/`contentctl-ng`, `panther_analysis_tool`, `suricata.yaml`24and `*.rules`, `*.zeek` scripts, Wazuh `local_rules.xml`/`decoders`, `falco_rules.local.yaml`,25ATT&CK Navigator `.json` layers, `chainsaw`, `hayabusa`, `atomic-red-team`, "detection26rule", "SIEM use case", "false positive", "tuning", "ATT&CK coverage", "MTTD".2728**Guiding principle**: detection is an **engineering product with a lifecycle**, not a29catalogue of bought alerts. It is governed by three invariants: **(1) without the right source there is30no detection** — the data model precedes the rule; **(2) every rule is tested with the31real attack it claims to detect** before it is believed; **(3) an alert with no possible action is not32an alert, it is noise with an owner**. Uncomfortable corollary: **coverage ≠ detection**. "We cover33300 techniques" almost always means "we have 300 rules nobody has run against an34attack".3536**Not applicable**: see `observability-standards` (telemetry for **diagnosing**: OpenTelemetry,37Prometheus, Loki, cardinality, cost, Alertmanager — **the boundary is the purpose, not the38tool**: the same log feeds both, there to explain an outage, here to discover39an adversary; the pipeline, retention and integrity of the telemetry are theirs, the40analytical content is mine), `sre-practice-standards` (on-call, SLOs, error budget, burn rate,41postmortems), `incident-management-standards` (**governance of the incident my alert42triggers**: declaration, severity, IC, communication — my deliverable ends exactly at the43handoff, and its retro sends work back to me), `incident-response-forensics-standards` (**technical44response and investigation**: containment, imaging, timeline, eradication — a45**bidirectional** boundary: there they investigate what I detect, and **every investigation must come back46turned into a new rule**; the use of YARA for forensic *hunting* and evidence triage is47theirs, the **authoring and governance of the rule** is mine), `offensive-security-standards` (purple48team, adversary emulation, Atomic Red Team and Caldera as **generators** of the authorised49attack and its deconfliction with the SOC — **they generate the activity, I write and validate50the detection**), `container-runtime-security-standards` (**which runtime signal matters** —51unexpected shell, writes to binaries, runtime socket, drift— and the deployment of the52eBPF agent: **the Falco rule is born there and is governed here**, with my lifecycle, my tests53and my coverage), `linux-hardening-standards` and `selinux-standards` (CIS baseline, `auditd` and54SELinux AVCs **as a source**: configuring the audit ruleset is theirs, turning55those events into detection is mine), `windows-server-ad-standards` (which AD event matters and56why), `networking-standards` (segmentation, NetFlow/IPFIX and the network Suricata/Zeek57observe), `appsec-standards` (**A09 Security Logging and Alerting Failures**: which events58the application must **emit** — authn, denied authz, privilege change; I consume that59emission, I do not design it), `vulnerability-management-standards` (CVE, CVSS/EPSS/KEV and patching60SLAs), `privacy-engineering-standards` (PII inside security logs, minimisation61and retention limits by purpose), `secrets-management-standards` (custody and rotation of62secrets — **shared boundary**: a leaked secret or a secret used from an anomalous origin is a63**detection case of mine**, fed by their manager and scanning telemetry), `grc-compliance-standards` (the64regulatory control requiring monitoring and its evidence), `identity-access-management-standards` (IdP design,65conditional access policies and OAuth flows — I detect their abuse, I do not configure them),66`threat-intelligence-standards` (**bidirectional boundary**: the hypothesis, the indicator and its67expiry are theirs, the rule is mine — and **the return of *sightings* is theirs**: an indicator that68expires for lack of sightings is retired there, by their criteria, not here).6970## 2. Default decisions7172> Data verified Aug 2026. **Verify the latest version, licence and status on the web before73> pinning it in a real project (§8)**: this ecosystem rotates in months and several projects74> changed governance in 2025-2026.7576| Area | Default | Justifiable alternative / vetoed |77|---|---|---|78| Portable authoring format | **Sigma** — spec **v2.1.0** (Sep 2025, formalises *correlations* and *meta-rules*), `pySigma` **1.5.0**, `sigma-cli` **3.1.0**, `SigmaHQ/sigma` rules **r2026-07-01** | Writing directly in the SIEM's language **only** when translation loses semantics (§3). **Hard minimum: pySigma ≥ 1.3.0** — earlier versions executed code via *template vars* (today behind `--enable-template-vars` and a sandboxed Jinja2) |79| Files and memory | **YARA-X 1.19.0** | Classic YARA **4.5.8** only for third-party engine compatibility: VirusTotal declared it in **maintenance mode** when publishing YARA-X 1.0.0 (Jun 2025) and new modules (`macho`, `lnk`) only reach YARA-X. Migrate while validating the ruleset (≈99 % compatible at rule level) |80| Threat reference framework | **MITRE ATT&CK v19.1** (Apr 2026) + **Navigator 5.3.2** | None. It is the lingua franca. Watch two breaks: **v18.0** replaced *Detections* with **Detection Strategies (`DETxxxx`) + Analytics (`ANxxxx`)** and deprecated the legacy *Data Sources*; **v19** split **Defense Evasion** into **Stealth (TA0005)** and **Defense Impairment (TA0112)** — use the official *crosswalk*, do not renumber by hand |81| Coverage measurement | **DeTT&CT 2.2.0** over Navigator layers, scoring **data source quality**, not just rule presence | Your own spreadsheet (it desynchronises on its own). **Vetoed**: presenting "techniques with a rule" as coverage without scoring visibility |82| Adversarial validation | **Atomic Red Team** (MIT, *rolling* model with no tags) for atomic tests per technique | **Apache Caldera** for chains and campaign emulation — **it changed governance: MITRE contributed it to the Apache Incubator on 20 May 2026**, canonical repo `apache/caldera`. **Hard minimum ≥ 5.1.0** because of **CVE-2025-27364** (CVSS 10.0, preauth RCE in agent compilation) |83| Normalisation schema | **The one of the engine you consume**, declared explicitly: **ECS 9.4.0** in the Elastic/Sigma world; **OCSF 1.8.0** (Linux Foundation since Nov 2024) in a data lake and for multi-vendor interoperability | **There is no single winner today.** Frequent correction: **ECS was not donated to OCSF, but to OpenTelemetry (Apr 2023)**, and that convergence is still **incomplete**. Pick one per platform and write it down; translate at the edge, not in the rule |84| Low-cost self-hosted SIEM | **Wazuh 4.14.7** (GPLv2) | **Hard minimum 4.14.4+**. Wazuh has a serious history: **CVE-2025-24016** (CVSS 9.9, deserialisation in DAPI, fix 4.9.1) was **actively exploited by Mirai botnets**. It is a server exposed to agents: treat it as Tier 0 |85| Commercial SIEM | Whichever you already have; portability comes from Sigma, not the product | **Elastic Security 9.4.x** (**triple** licence: AGPLv3 / SSPL / Elastic License — verify which applies to your deployment); **Microsoft Sentinel**; **Google SecOps** (formerly Chronicle) with **YARA-L 2.0**; **Splunk ES** |86| Network — IDS/IPS | **Suricata** (8.0.6 and 7.0.17 in parallel, GPLv2) for signatures and protocol detection | **Zeek 8.0.9** (BSD) for network **metadata and context**: they do not compete, they complement each other. Zeek gives you the `conn.log`/`ssl.log` that makes the Suricata alert investigable |87| Container runtime | **Falco 0.44.1** (Apache-2.0, CNCF **graduated** since Feb 2024) | The choice of agent and the *what to watch* belongs to `container-runtime-security-standards`; here the content lifecycle |88| Offline EVTX triage | **Hayabusa 3.10.0** (AGPL-3.0, ATT&CK v19) or **Chainsaw 2.16.2** (GPL-3.0) — both run Sigma over evidence | Manual EVTX parsing. Also useful as a **test engine** for Windows Sigma rules in CI |89| Third-party content as a base | **SigmaHQ** (rules), **elastic/detection-rules**, **splunk/security_content (ESCU 6.3.0, Apache-2.0)**, **panther-analysis 3.112.0** (Apache-2.0), **Nextron `signature-base`** (**DRL 1.1** licence — check before using in a commercial product) | Consuming the whole package without calibrating (§7). **Vetoed**: closed vendor rules whose logic you can neither read nor version |9091**Projects not to choose in 2026** (verified): **Matano** — no commits since Jan 2025;92**DetectionLab** — declared dead by its author since 2023. **Panther**: Databricks announced an93acquisition agreement on 16 Jun 2026 and **the release says nothing about the future of the OSS** —94`panther-analysis` is still Apache-2.0 and active, but it is a bet with governance risk.95Worth mentioning in case it applies to the European public sector: **OpenTIDE** (European Commission, EUPL 1.2,96org `OpenTideHQ`) is today the most complete formal open-source detection-as-code framework.9798## 3. Structure and conventions99100### 3.1 The lifecycle (a rule is a software artifact, not a ticket)101102`idea/hypothesis → data source → rule → test → deployment by CI → measurement → tuning → retirement`103104- **Idea/hypothesis**: every rule is born from a written hypothesis (`"an attacker with valid105 credentials will register a new device in Entra after device code phishing"`), with an106 ATT&CK technique, a reference actor or campaign, and **expected value**. Without a hypothesis there is no107 criterion for knowing whether the rule works or for retiring it.108- **Source before rule** (§3.2): if the event does not arrive, the rule is theatre.109- **Rule**: in the repo, with complete metadata (§3.4), reviewed in a PR by somebody other110 than the author.111- **Test**: mandatory unit test (§4), plus adversarial validation before raising the severity.112- **Deployment by CI**: never by hand in the SIEM console. The console is *read-only* for113 content; what is edited there is lost at the next sync or produces invisible114 *drift*.115- **Measurement**: FP rate, firings, actions taken. A rule with no telemetry about itself116 cannot be governed.117- **Retirement**: a rule that has fired nothing useful in its review window **is deleted or118 downgraded to informational**, in the same PR that establishes this. The cost of a dead rule is119 not zero: it consumes compute, dirties the coverage and wears down the analyst who triages it.120121**Reference repo structure** (adapt it, but have one):122123```124detections/125 rules/<platform>/<tactic>/<id>.yml # Sigma or the engine's native format126 pipelines/<platform>.yml # pySigma processing pipelines127 tests/<id>/{positive,negative}.jsonl # events that MUST and MUST NOT fire128 runbooks/<id>.md # mandatory (§3.5)129 coverage/navigator-*.json # generated layers, not hand-edited130 deprecated/ # with reason and date, not silent deletion131```132133### 3.2 Data model and telemetry: the part almost nobody does134135- **Onboarding a source is a project, not a checkbox.** For every source:136 *which concrete events* (by ID/category, not "the AD logs"), *how they are transported*,137 *how they are normalised*, *how long they are retained*, *what they cost* and *which detections they enable*.138 If it enables none, it is not ingested.139- **Cost and retention are first-class design constraints**, exactly as in140 `observability-standards`. Healthy pattern: **short hot retention (interactive search) +141 long, cheap cold retention (investigation and compliance)**. Practical rule: the cold142 window must cover the *dwell time* you assume, not the one you would like.143- **Normalise at the edge, not in the rule.** A rule that slices strings to reconstruct a144 field the pipeline should have normalised is debt paid on every new source.145- **Measure source health as if it were an SLI**: ingestion latency, volume per hour146 with an expected band, and **alert on the absence of events** — a source's silence is147 indistinguishable from "nothing happened", and it is exactly what an attacker who148 disables the agent produces. *Detection of missing detection* is the first rule you write.149- **Integrity**: security logs are kept separate from operational telemetry (different access150 control, retention and **immutability**). The SIEM is the record an attacker151 will want to erase.152153**Source priority today** (by value/cost ratio, not by tradition):1541. **Identity and the cloud control plane** — it is the main surface (§3.7).1552. **Endpoint/EDR** (process, command line, parent/child, module loading).1563. **Authentication and directory** (AD, Entra, IdP).1574. **CI/CD and the artifact registry** (§5).1585. **Network** (DNS, proxy/egress, `conn.log`): indispensable for investigating, mediocre for159 detecting on its own.160161### 3.3 ATT&CK coverage with honesty162163- Coverage is scored by **visibility quality × detection quality**, not by164 rule count. Minimum scale per technique: `no telemetry` → `telemetry with no rule` →165 `unvalidated rule` → `rule validated against a real execution` → `rule validated and with166 a response procedure`. Only the last two count as coverage.167- **A technique covered by a single variant is not covered.** ATT&CK enumerates168 behaviours, not commands: if the rule only matches the binary the atomic test used, you have169 covered the tool, not the technique.170- Prioritise by **relevant threat** (adversary profile, sector, real surface), not by171 filling the matrix. A uniformly green matrix is a sign the colour has been optimised.172- **Generate the Navigator layers from the repo in CI.** A hand-edited coverage map173 lies as soon as the next PR is merged.174- When jumping ATT&CK versions, **use the official crosswalk** (the split of Defense175 Evasion in v19 invalidates previous mappings) and treat the migration as a change with a PR and176 review, not as a `sed`.177178### 3.4 Anatomy of a rule that can be governed179180Mandatory metadata, no exceptions: a stable `id` (UUID, never reused), an actionable181`title`, `status` (`experimental` → `test` → `stable` → `deprecated`), a `description` with182the hypothesis, `author`/`owner` (a team, not a person), `date`/`modified`, a justified `level`,183ATT&CK `tags` with the framework version, `falsepositives` **populated with real observed184cases**, `references` and **a link to the runbook**.185186- **Severity is earned, not declared.** A rule is born in `experimental` with informational187 severity; it goes up only with data: N days in production, a measured FP rate, and188 adversarial validation passed. Raising the severity "because the technique is serious" is the direct route to189 alert fatigue.190- **Actionable context in the event itself**: who (a resolved identity, not a GUID), which191 asset (with owner and criticality), what happened (relevant raw command/fields), what is expected192 of the analyst. An alert that forces you to open four consoles to know whether it matters has already193 failed.194- **Enrichment in the pipeline, not in the analyst's head**: asset inventory,195 criticality, owner, geolocation, reputation, and **whether that activity was196 authorised** (change window, deconflicted red team exercise).197- **Deduplication and grouping**: the unit of work is the **incident**, not the alert. It is198 grouped by the entities that define *one and the same case* (identity, host, session, campaign) with199 a time window. **Sigma v2 correlations** (`event_count`, `value_count`,200 `temporal`) are the portable mechanism for expressing chains and thresholds without tying yourself to the engine.201- **Portability with a declared limit**: Sigma is the authoring format; the backend translates.202 When the semantics do not survive translation (complex correlation, joins, engine-specific203 functions, sequence analysis), **write it in the native language and declare204 the exception in the rule** — do not force a Sigma that translates into something other than what it says.205- **Freshness of pySigma backends: verify it, do not trust the label.** The206 official directory marks abandoned backends as *stable* (`sentinelone` unpublished207 since 2023, `ibm-qradar-aql` and `insightidr` equally behind). Before betting a208 platform on a backend, look at the date of the last release, not the badge.209210### 3.5 Mandatory runbook: a rule with no action is not deployed211212Every alert carries a runbook versioned **in the same repo and in the same PR**, with: what the213firing means, **how it is verified in 5 minutes** (concrete queries, not "investigate"), what is214known benign, what immediate action is appropriate (contain, revoke session, isolate,215escalate), to whom it is escalated and at what severity, and which data to preserve before touching anything216(preservation belongs to `incident-response-forensics-standards`; the link is mandatory).217218**If you cannot write the runbook, you do not know what the rule detects: it is not merged.**219220### 3.6 Adversarial validation and regression detection221222- **No rule leaves `experimental` without having fired against a real execution of the223 technique.** The attack generator belongs to `offensive-security-standards`; the evidence of224 the firing is my deliverable.225- **Purple team ≠ red team**: the value is not "they got in", it is the **executed226 vs. detected vs. alerted vs. responded** matrix by technique. Every undetected execution opens227 a backlog item with an owner and a date.228- **Regressions**: a rule that used to work stops working on its own when the agent changes,229 or the vendor's schema, the OS version or a binary's path. **Scheduled and automatic**230 execution of a subset of atomics against a controlled environment, with an alert if the expected231 detection does not appear. A real and recent example of why this is not theoretical: Defender's232 *advanced hunting* schema renamed `AADSignInEventsBeta` to233 `EntraIdSignInEvents` and **removed the legacy tables on 9 Dec 2025**, and on 25 Feb 2026 the234 booleans went from `1`/`0` to `True`/`False` — each of those changes breaks rules235 silently.236- Every adversarial execution is **deconflicted with the SOC** beforehand: an exercise that consumes the237 real incident process is a process failure, not a test passed.238239### 3.7 Identity and cloud: where the detection that matters lives today240241The perimeter is identity. The minimum sources, with **exact names** (verify §8):242243- **Entra ID**: `SigninLogs` (interactive), `AADNonInteractiveUserSignInLogs`,244 `AADServicePrincipalSignInLogs`, `AADManagedIdentitySignInLogs`, `AuditLogs` and245 **`MicrosoftGraphActivityLogs`** (requires P1/P2). In 2026 **Agent logs** were added for246 *Entra Agent ID*, and `AADNonInteractiveUserSignInLogs` now carries an `Agent` column: AI247 agent identities are a new surface and must be ingested from day one.248- **AWS CloudTrail**: there are now **four** categories, not two — *Management*, *Data*, ***Network249 activity*** (`eventCategory = NetworkActivity`; the only valid `errorCode` is250 `VpceAccessDenied`, useful for seeing credentials outside your organisation using your VPC251 endpoints) and *Insights*. **Only management is on by default**: data and network events252 are enabled explicitly and budgeted for.253- **GCP**: `cloudaudit.googleapis.com%2Factivity` (Admin Activity, always),254 `%2Fdata_access` (**disabled by default** except for BigQuery), `%2Fsystem_event`,255 `%2Fpolicy`. For consent abuse in Workspace, the key log is the **OAuth Token256 Audit** (`applicationName=token`, `authorize`/`revoke` events, with `scope_data`,257 `client_id`, `app_name`); default retention **6 months** — if your investigation258 window is longer, export it.259- **GuardDuty Extended Threat Detection** is on by default and at no extra cost, with a rolling260 24 h window. **Verified operational trap: it ignores archived findings,261 including those archived by your own suppression rules** — aggressive tuning blinds you to262 attack sequence correlation.263264**Mandatory identity use cases** (defensible ATT&CK chain:265`T1566.002 → T1528 → T1550.001 → T1098.005 / T1098.001`; **there is no dedicated sub-technique for266device code phishing** — the one that mentions it explicitly is T1566.002, not T1528):267268- **Device code phishing**: the decisive field in Entra is `AuthenticationProtocol == "deviceCode"`.269 It is the technique of **Storm-2372** (Microsoft, Feb 2025; **reclassified on 31 Jul 2026 as a270 sub-cluster of Midnight Blizzard/SVR**), which pivoted to the client ID271 `29d9ed98-a469-4536-ade2-f981bc1d605e` (*Microsoft Authentication Broker*) to obtain a272 **PRT** and register a device. Preventive control: the Conditional Access **"Authentication flows"**273 condition. Planning notice: Microsoft is rolling out a **managed policy274 "Block device code flow"** which it creates in *report-only* and **enables on its own after a minimum of 45275 days**, warning 28 days in advance; audit it by filtering `AuditLogs` on the initiator276 `Microsoft Managed Policy Manager`. Critical caveat: **a CA policy targeting users277 does not cover service principals**.278- **Device registration and new credential**: adding or changing credentials on279 *service principals* and *app registrations*. It is exactly the pattern of the280 **Commvault/Metallic** case (CISA, May 2025, **CVE-2025-3928** in KEV): theft of *client secrets*281 from a cross-tenant app.282- **Abuse of a third-party integration's OAuth token**: the **Salesloft Drift / UNC6395** case283 (Aug 2025) exfiltrated Salesforce data with legitimate OAuth tokens, searching with mass284 SOQL for strings such as `AKIA`, `Snowflake`, `password`, `secret`, and **deleting the records of285 the query jobs**. Signals: Salesforce Event Monitoring `UniqueQuery` events, UA286 `Salesforce-Multi-Org-Fetcher/1.0`. Design lesson: **a third-party app's consent287 is a permanent key with no MFA** — inventory and watch consents the way you288 watch administrators.289- **Impossible travel and session anomaly**: useful, but **only as enrichment or a low-severity290 signal**. In a workforce with a corporate VPN and mobiles it generates massive FPs; never291 as a paging alert on its own.292- **Graph↔sign-in correlation** (verified canonical pattern):293 `MicrosoftGraphActivityLogs | join ... on $left.SignInActivityId == $right.UniqueTokenIdentifier`294 — it is what turns "a token did things" into "this session, of this user, from this295 origin, did these things".296297### 3.8 Threat intelligence: pyramid of pain, not shopping list298299- **IoCs expire; TTPs do not.** Hashes and IPs are rotated in hours and their value is almost purely300 retrospective; tools and TTPs cost the adversary and survive months or years. The301 engineering effort goes to the **top** of the pyramid; IoCs are automated and forgotten.302- **Every IoC comes in with an expiry and a provenance.** A feed with no expiry date is a303 source of FPs that grows on its own. Indiscriminate free feeds generate more triage than they304 save: they are measured like any rule (§6) and cut if they do not pay off.305- **Systematic retro-hunt**: every new IoC is searched **backwards** in the cold retention, not306 only forwards. That is where the IoC does earn its keep.307- The ultimate goal of TI is not the list: it is **prioritising the detection backlog** by308 relevant adversary.309310## 4. Quality gates (they break the build)311312In increasing order of cost. The first four are blocking on every detection PR.3133141. **Lint and schema**: the rule parses, validates against the format's schema (Sigma spec 2.1,315 `detection-rules` TOML, the engine's schema) and **compiles with the target backend**316 (`sigma convert`). A rule that does not translate does not exist.3172. **Complete metadata**: a unique, non-reused `id`, owner, `status`, ATT&CK tags318 **existing in the declared version** of the framework, non-empty `falsepositives`.3193. **Runbook present and linked.** No runbook, no merge (§3.5). Non-negotiable.3204. **Unit test with positives and negatives**: at least one event that **must** fire and321 several that **must not** (the rule's own known FPs). Valid engines:322 `detection_rules test` (Elastic), `panther_analysis_tool test`, Hayabusa/Chainsaw over323 sample EVTX, or the backend's runner. **A rule with no negative test is not tested:324 it is declared.**3255. **Schema non-regression test**: the test events are validated against the current326 schema (ECS/OCSF/vendor table), so that a field rename breaks the build327 and not the detection in silence (§3.6).3286. **Coverage**: the Navigator layer is **regenerated in CI** and the diff is reviewed. Adding a329 rule that does not move the declared coverage is a sign of a duplicate.3307. **Adversarial validation** before promoting to `stable` or raising severity: a real331 atomic execution and evidence of the firing attached to the PR.3328. **Deployment by pipeline** with versioning and rollback: `Sentinel repositories` (GA333 Mar 2026), `elastic/detection-rules` with a *version lock* per stack branch, or the equivalent334 mechanism. **Manual deployment in the console is forbidden.**335336**Measurable promotion rule** (adjust the numbers, but fix some): `experimental` → `test`337after compiling and passing tests; `test` → `stable` after ≥ 2 weeks in production with a **measured338FP rate < 20 %** and adversarial validation passed. **It is measured before raising the severity,339never after.**340341## 5. Security of the detection programme itself342343- **The supply chain of security tooling is a first-order target, and 2026344 proved it.** The compromise of **Trivy** by the actor *TeamPCP* (**CVE-2026-33634**, CVSS345 9.4, in **CISA KEV** since 26 Mar 2026) went as far as **force-pushing 76 of 77 tags** of346 `trivy-action` and all 7 of `setup-trivy`, with a payload that **read the memory of the347 `Runner.Worker` process to extract masked secrets**; the same campaign hijacked the 35 tags348 of `Checkmarx/kics-github-action`. Analogous precedent: `tj-actions/changed-files`349 (**CVE-2025-30066**). Operational consequences, without exception:350 - **Pin by commit SHA** every GitHub Action in the detection pipeline; **pin by351 digest** every image; verify signatures and checksums of the detection binaries.352 - **Immutable releases** enabled in your own content repos.353 - **Do not confuse provenance with goodness**: the *CanisterWorm* variant went as far as **generating354 valid SLSA Build L3 attestations via Sigstore for malicious packages**. The signature355 proves **who** built it, not **what** it does.356 - The detail of pipeline hardening belongs to `cicd-standards`; here it is an entry357 requirement.358- **Verified Aug 2026: none of the recommended detection tools (Sigma,359 YARA/YARA-X, Atomic Red Team, Caldera, Falco, Suricata, Zeek, Wazuh, Elastic) appears as a360 victim of a repository compromise in 2025-2026.** The vector, however, applies to them361 just the same: do not treat it as ruled out, but as not having happened yet.362- **The SIEM and its rules are sensitive data.** The ruleset reveals exactly what you see and what363 you do not: access control on the detection repo, review of who can disable or modify364 rules, and **an alert on the modification or silencing of the content itself**. Disabling365 a rule is a security event.366- **Never embed secrets in rules or in queries** (API tokens, enrichment367 credentials, webhook keys). They go to the secrets manager368 (`secrets-management-standards`).369- **PII and minimisation**: security logs contain personal data by definition.370 Legal basis, retention by purpose and access control belong to371 `privacy-engineering-standards`; the requirement here is that **the rule and its enrichment do not372 widen the set of personal data** beyond what is needed to decide.373- **Integrity and non-repudiation** of the evidence: append-only logs, a synchronised and374 verified clock (clock drift destroys any temporal correlation and any375 forensic timeline) and control over who can purge.376377## 6. Operability, cost and programme metrics378379**Metrics that are published (and what corrupts them)**380381| Metric | What it measures | How it gets corrupted |382|---|---|---|383| **MTTD** | Time from the adversary's activity to detection | Measuring from ingestion, not from the event; excluding what was never detected (survivorship bias). It is only honest with purple team, where you know the real `t0` |384| **Coverage per source** | % of assets of each class reporting the expected source | Counting configured sources instead of sources **that are sending events now** |385| **FP rate per rule** | Firings closed as benign / total firings | Not recording the analyst's disposition; then there is no data and tuning is opinion |386| **Alerts per analyst per hour** | Real triage load | Counting alerts instead of **grouped incidents**. An alarm threshold for the programme, not for the shift |387| **Orphaned rules** | No owner, no firings, no runbook, no review within its window | None: it is the metric hardest to dress up and the one that best predicts the programme's collapse |388| **% of adversarially validated rules** | Real coverage | Counting written rules |389390- **Alert fatigue is the domain's modal failure.** If the analyst ignores the queue,391 detection does not exist even if the rules are perfect. Work priority when the queue392 overflows: **reduce noise before adding coverage**, always.393- **Tuning ≠ disabling.** The correct order is: (1) fix the data or the enrichment,394 (2) narrow the rule's condition, (3) a **narrow, documented exception with an395 expiry**, (4) downgrade to informational, (5) retire. A broad and permanent suppression is396 a detection hole shaped like tuning — and on AWS it also blinds GuardDuty's397 correlation (§3.7).398- **Cost**: the SIEM is paid for by ingestion, by retention or by query compute depending on the399 product — **find out your billing unit before designing**, because it decides what to optimise.400 Levers in order: do not ingest what does not enable detection; filter and aggregate at the edge;401 tiered retention to cheap storage; efficient scheduled queries. Ingesting "just in402 case" is the most expensive way of detecting nothing.403- **Rule performance**: scheduled queries with a coherent window and frequency404 (window ≥ frequency + ingestion latency, or you lose events at the edge), no joins405 over the whole history, no catastrophic regexes. A rule that does not finish within its406 interval does not detect: it skips executions.407- **Platform change with a hard date**: if you use **Microsoft Sentinel**, the product is already408 GA in the Defender portal (Jan 2026) and **stops being supported in the Azure portal409 after 31 Mar 2027**. It is a migration project with a date, not a UI change.410411## 7. Sustainability and prohibitions412413- **Cadence**: review ATT&CK and remap on every major framework release (~half-yearly;414 v18 and v19 brought structural breaks); review the entire ruleset **quarterly** with415 explicit pruning; update `pySigma`/backends and the community ruleset monthly;416 review sources and their health continuously.417- **Every exception and every suppression carries an expiry and an owner.** Without a date, they become418 permanent holes nobody remembers opening.419- Third-party content is **calibrated on adoption**, not "some day": it enters the repo, passes420 the gates of §4 and is assigned an owner, or it does not enter.421- When a project changes governance (Caldera → Apache Incubator; Panther → Databricks),422 **review it as a platform risk**, not as a press release: licence, release cadence423 and who responds to a CVE.424425**FORBIDDEN**426- ❌ Deploying a rule **without a unit test** (positive **and** negative).427- ❌ Deploying a rule **without a runbook** with a concrete action.428- ❌ **Raising the severity without having measured** the FP rate in production.429- ❌ Creating or editing content **directly in the SIEM console**, with no repo and no PR.430- ❌ Enabling the product's default rule pack **without calibrating** — the domain's central431 antipattern: a SIEM full of rules nobody has touched is not coverage, it is a432 noise queue that guarantees the good alert goes unnoticed.433- ❌ Presenting a **count of rules or techniques** as coverage without scoring visibility and434 validation.435- ❌ Rules with no owner, no stable `id`, no ATT&CK version tags or with empty436 `falsepositives`.437- ❌ Writing rules for a source **not confirmed in ingestion**, or leaving the **absence of events**438 from a source without an alert.439- ❌ Broad, permanent or ownerless suppressions; silencing a rule instead of fixing it.440- ❌ Alerting on IoCs with no expiry or provenance, or building the programme on IoCs instead441 of on TTPs.442- ❌ Impossible travel (or other pure anomaly heuristics) as a standalone paging alert.443- ❌ Consuming GitHub Actions or images in the detection pipeline **by mutable tag** instead of444 by SHA/digest, or treating an SLSA attestation as proof that the artifact is benign.445- ❌ Running adversarial validation **without prior deconfliction** with the SOC.446- ❌ Secrets or credentials embedded in rules, queries or enrichment pipelines.447- ❌ Starting anything new on **Matano** or **DetectionLab** (unmaintained), or on classic448 YARA when YARA-X is available.449- ❌ Running Wazuh or Caldera on versions earlier than the minimums of §2 (critical CVEs, one450 of them mass-exploited).451- ❌ Forcing into Sigma a logic the backend does not support and trusting the452 result.453454## 8. Mandatory web verification455456Before pinning any version, licence, project status or table/field name,457**look it up — do not recall it**. Data verified Aug 2026 (it expires fast): Sigma spec458**v2.1.0**, pySigma **1.5.0**, sigma-cli **3.1.0**, SigmaHQ rules **r2026-07-01**; ATT&CK459**v19.1** (Apr 2026) and Navigator **5.3.2**; ATT&CK Workbench **4.10.0**; OCSF **1.8.0** (LF460since Nov 2024); ECS **9.4.0**; Elasticsearch **9.3.0** / Elastic Security **9.4.4**; Wazuh461**4.14.7**; Suricata **8.0.6** and **7.0.17**; Zeek **8.0.9**; Falco **0.44.1** (CNCF graduated462Feb 2024); YARA-X **1.19.0** and YARA **4.5.8**; Splunk ESCU **6.3.0**; `panther-analysis`463**3.112.0**; Chainsaw **2.16.2**; Hayabusa **3.10.0**; DeTT&CT **2.2.0**.4644651. **The current ATT&CK version and its changelog**, including the *crosswalk* for the split of466 Defense Evasion (v19) and the Detection Strategies/Analytics model (v18) — **before**467 remapping anything.4682. **The real freshness of the pySigma backend** you are going to use: the date of the last release in the469 `pySigma-plugin-directory`, not the `stable` badge.4703. **Governance and health** of Caldera (Apache Incubator since May 2026, **current version not471 verified here**) and of Panther (acquisition by Databricks announced Jun 2026, **future472 of the OSS not declared**).4734. **Exact names of the vendor's tables, fields and logs** before writing the query:474 the Defender/Entra, CloudTrail and Cloud Audit Logs schemas changed in 2025-2026 and475 keep changing.4765. **Hard platform dates**: end of support for Sentinel in the Azure portal477 (31 Mar 2027, verify), status of the content migrations under way.4786. **Advisories and CVEs** for your SIEM and your agents (Wazuh and Elastic have had recent479 criticals) and for the tooling you run in CI.4807. **Supply chain compromises** of any new action, image or binary in the481 detection pipeline before adopting it (§5).482483**Declared gaps — not verified in this document, verify them yourself before using them**:484- **The current version of Apache Caldera (incubating)**.485- **The version of Splunk Enterprise Security** and product news after the acquisition by486 Cisco (the official documentation was not accessible during verification).487- **Formal EOL dates for the Suricata 7.0 and 8.0 branches**, and whether a 9.0 release exists or is488 planned (the upgrade guide mentions it; no release has been published).489- **The status of OCSF support in Elastic Security and in Splunk** in 2026.490- **Whether Red Canary (maintainer of Atomic Red Team) is today part of Zscaler** — a secondary491 source, unconfirmed against a primary release.492- **GA dates** for the Conditional Access "Authentication flows" condition, for CloudTrail's493 *network activity events* and for GuardDuty Extended Threat Detection.494- **Secondary ATT&CK IDs** cited in identity literature (T1621, T1539, T1550.004,495 T1078.004, T1098.002/.003, T1606.002): not individually verified against the site.496- **Whether the recommended detection projects consumed** `trivy-action` or497 `tj-actions/changed-files` during their exposure windows: not ruled out, just not498 found.499500If the web contradicts this document, **the web wins** — flag the discrepancy.