Logging Architecture
Plan centralized logging, retention, and audit trail design for "$ARGUMENTS". Define log sources, standardize formats, design aggregation infrastructure, and establish retention policies aligned with compliance requirements.
Arguments. $ARGUMENTS may contain the project name and optional flags. Take the first token as <project> and treat --skip-checks as a flag; never include flags in file paths. All paths below use <project>. If --skip-checks is present, skip the prerequisite check and, before writing any artifact, append one line to .singularskills/log.jsonl: {"ts":"<ISO timestamp>","skill":"security-logging-architecture","workflow":"security","phase":3,"project":"<project>","event":"skipped","artifact":"03-detect.md","note":"<reason or \"none given\">"}.
Prerequisites
Part of a workflow. This skill is phase 3 (Detect) of the NIST CSF workflow (security); cross-domain workflows run it too. First check .singularskills/log.jsonl for an open run for this project: a run-started line with "project":"<project>" and no later run-finished line for the same workflow and project. If one exists, the conductor is running you: continue here. Otherwise, if the user asked to start, run, continue or resume a process or workflow rather than this one step, do not continue here: invoke the workflow skill with the workflow the user asked for, or security when they named none, followed by <project>; it runs this phase in order, asks before it writes and records the run. Otherwise continue here: a later phase runs on its own, behind its gate.
Run this skill's gate before writing: sh <this skill's folder>/scripts/gate.sh <project> <project root>, where <project root> is the folder that contains .singularskills/ (use . when you are already there). If it exits non-zero, stop and tell the user what it printed, unless --skip-checks was given (see Arguments). On a refusal, before you stop, append this line to .singularskills/log.jsonl (create the folder and file if missing): {"ts":"<ISO timestamp>","skill":"security-logging-architecture","workflow":"security","phase":3,"project":"<project>","event":"refused","artifact":"03-detect.md","note":"<what the gate printed>"}. Once the gate has passed, or under --skip-checks once you have appended the skipped line, append this line before you start work: {"ts":"<ISO timestamp>","skill":"security-logging-architecture","workflow":"security","phase":3,"project":"<project>","event":"started","artifact":"03-detect.md","note":""}.
Read .singularskills/security/<project>/00-govern.md. If this file does not exist, tell the user:
Phase 0 (Govern) has not been completed for "". Run a Govern skill first (e.g., /security-security-policy <project>), or use --skip-checks to bypass.
Process
Define log sources:
- Application logs — request/response logs, business logic events, error logs
- Infrastructure logs — server logs, container logs, load balancer logs, CDN logs
- Security tool logs — WAF, IDS/IPS, endpoint protection, vulnerability scanners
- Cloud provider logs — CloudTrail, Azure Activity Log, GCP Audit Logs, VPC flow logs
- Database logs — query logs, slow query logs, access logs, replication logs
- Identity provider logs — authentication events, token issuance, MFA events
Standardize log format:
- Adopt structured logging format (JSON recommended)
- Define mandatory fields: timestamp (ISO 8601 UTC), service name, environment, log level, correlation ID, user ID (when applicable), source IP
- Implement correlation IDs across service boundaries for request tracing
- Sanitize logs to prevent sensitive data in log entries (PII, credentials, tokens)
- Define log level conventions (ERROR, WARN, INFO, DEBUG) with usage guidelines
Design centralized log aggregation:
- Evaluate aggregation platforms (ELK/OpenSearch, Splunk, CloudWatch Logs, Datadog, Sumo Logic)
- Design collection architecture (agents, sidecars, log forwarding, direct API ingestion)
- Plan for log volume estimation and capacity planning
- Implement log parsing and enrichment (geo-IP, threat intelligence lookup)
- Design search and query capabilities for incident investigation
Define retention policy:
- Security logs — 1 year minimum (authentication, authorization, access control events)
- Audit logs — 7 years for compliance (SOX, PCI DSS, HIPAA as applicable)
- Application logs — 90 days hot storage, 1 year cold storage
- Infrastructure logs — 30 days hot storage, 6 months cold storage
- Define storage tiers (hot/warm/cold/archive) with cost optimization
- Set automated lifecycle policies for tier transitions and deletion
Protect log integrity:
- Store logs in immutable storage (write-once, read-many) for audit trail
- Implement checksums or digital signatures for log entries
- Restrict access to log data (separate from application access controls)
- Ensure log deletion requires multi-party approval
- Design tamper-evident log chains for high-security requirements
Define access control for log data:
- Restrict log access by role (security team: full access, developers: own service logs, auditors: read-only)
- Implement PII masking in log views for non-privileged users
- Audit log access itself (who searched what, when)
- Define self-service log access for developers within guardrails
Write the artifact — append to .singularskills/security/<project>/03-detect.md under a ## Logging Architecture section. Create the file with a # Detect — <project> heading if it does not exist; if a ## Logging Architecture section already exists, replace only that section. The section contains:
- Log Sources — all sources with type, volume estimate, and sensitivity
- Log Format Standard — mandatory fields, correlation ID strategy, and sanitization rules
- Aggregation Architecture — platform selection, collection design, and capacity plan
- Retention Policy — retention periods by log type with storage tier strategy
- Log Integrity — immutability, checksums, and tamper protection measures
- Access Control — role-based log access and PII masking approach
Output
The logging architecture appended to .singularskills/security/<project>/03-detect.md. Present a summary to the user highlighting:
- Log sources to be aggregated and estimated volume
- Chosen aggregation platform and architecture
- Retention periods by log type and compliance alignment
- Log integrity and access control measures
Record the run. After you have written your section, append this line to .singularskills/log.jsonl (the started line of Prerequisites, with "event":"finished"): {"ts":"<ISO timestamp>","skill":"security-logging-architecture","workflow":"security","phase":3,"project":"<project>","event":"finished","artifact":"03-detect.md","note":""}.
This artifact is shared by every skill of phase 3 (Detect). The frontmatter is written once, by whoever creates the file: if you create it, open it with the block below (skill is the phase title, started the time of that first write; gate is passed when the gate ran clean and skipped under --skip-checks); if it already exists, leave the block alone except to update finished and to set gate to the worst value among the sections (skipped over passed).
---
skill: Detect
workflow: security
phase: 3
project: <project>
started: <ISO timestamp>
finished: <ISO timestamp>
agent: <the agent you run in, or "unknown">
gate: <passed|skipped>
---
Open your ## Logging Architecture section with one italic line, *<skill name> · <ISO timestamp> · gate: <passed|skipped>* (your skill name is security-logging-architecture).
End your section with ### Decisions and ### Open questions (one level under your ## Logging Architecture heading, so the section stays one block), in this shape:
### Decisions
- <what was chosen> — <why> — <what was rejected>
### Open questions
- <what the next phase or a reviewer must settle>
End the artifact with this paragraph, verbatim:
Not a penetration test or audit. This artifact is generated analysis; validate findings against the live system and have material controls reviewed by a qualified security professional.
1---2name: security-logging-architecture3description: Plan centralized logging — log sources, a structured format with correlation IDs, the aggregation platform, retention by log type, tamper-proof audit trails and access to log data. Use when the user asks how to centralize their logs, how long to keep them, or needs an audit trail that auditors will trust. Not for a SIEM product rollout; see /security-siem-setup.4metadata:5 source: https://skillmd.com/skills/bromso/security-logging-architecture6 argument-hint: "[project] [--skip-checks]"7 workflow: security8 phase: 39 artifact: 03-detect.md10---1112# Logging Architecture1314Plan centralized logging, retention, and audit trail design for "$ARGUMENTS". Define log sources, standardize formats, design aggregation infrastructure, and establish retention policies aligned with compliance requirements.1516**Arguments.** `$ARGUMENTS` may contain the project name and optional flags. Take the first token as `<project>` and treat `--skip-checks` as a flag; never include flags in file paths. All paths below use `<project>`. If `--skip-checks` is present, skip the prerequisite check and, **before writing any artifact**, append one line to `.singularskills/log.jsonl`: `{"ts":"<ISO timestamp>","skill":"security-logging-architecture","workflow":"security","phase":3,"project":"<project>","event":"skipped","artifact":"03-detect.md","note":"<reason or \"none given\">"}`.1718## Prerequisites1920**Part of a workflow.** This skill is phase 3 (Detect) of the NIST CSF workflow (`security`); cross-domain workflows run it too. First check `.singularskills/log.jsonl` for an open run for this project: a `run-started` line with `"project":"<project>"` and no later `run-finished` line for the same `workflow` and `project`. If one exists, the conductor is running you: continue here. Otherwise, if the user asked to start, run, continue or resume a process or workflow rather than this one step, do not continue here: invoke the `workflow` skill with the workflow the user asked for, or `security` when they named none, followed by `<project>`; it runs this phase in order, asks before it writes and records the run. Otherwise continue here: a later phase runs on its own, behind its gate.2122Run this skill's gate before writing: `sh <this skill's folder>/scripts/gate.sh <project> <project root>`, where `<project root>` is the folder that contains `.singularskills/` (use `.` when you are already there). If it exits non-zero, stop and tell the user what it printed, unless `--skip-checks` was given (see Arguments). On a refusal, before you stop, append this line to `.singularskills/log.jsonl` (create the folder and file if missing): `{"ts":"<ISO timestamp>","skill":"security-logging-architecture","workflow":"security","phase":3,"project":"<project>","event":"refused","artifact":"03-detect.md","note":"<what the gate printed>"}`. Once the gate has passed, or under `--skip-checks` once you have appended the `skipped` line, append this line before you start work: `{"ts":"<ISO timestamp>","skill":"security-logging-architecture","workflow":"security","phase":3,"project":"<project>","event":"started","artifact":"03-detect.md","note":""}`.2324Read `.singularskills/security/<project>/00-govern.md`. If this file does not exist, tell the user:2526> Phase 0 (Govern) has not been completed for "<project>". Run a Govern skill first (e.g., `/security-security-policy <project>`), or use `--skip-checks` to bypass.2728## Process29301. **Define log sources:**31 - **Application logs** — request/response logs, business logic events, error logs32 - **Infrastructure logs** — server logs, container logs, load balancer logs, CDN logs33 - **Security tool logs** — WAF, IDS/IPS, endpoint protection, vulnerability scanners34 - **Cloud provider logs** — CloudTrail, Azure Activity Log, GCP Audit Logs, VPC flow logs35 - **Database logs** — query logs, slow query logs, access logs, replication logs36 - **Identity provider logs** — authentication events, token issuance, MFA events37382. **Standardize log format:**39 - Adopt structured logging format (JSON recommended)40 - Define mandatory fields: timestamp (ISO 8601 UTC), service name, environment, log level, correlation ID, user ID (when applicable), source IP41 - Implement correlation IDs across service boundaries for request tracing42 - Sanitize logs to prevent sensitive data in log entries (PII, credentials, tokens)43 - Define log level conventions (ERROR, WARN, INFO, DEBUG) with usage guidelines44453. **Design centralized log aggregation:**46 - Evaluate aggregation platforms (ELK/OpenSearch, Splunk, CloudWatch Logs, Datadog, Sumo Logic)47 - Design collection architecture (agents, sidecars, log forwarding, direct API ingestion)48 - Plan for log volume estimation and capacity planning49 - Implement log parsing and enrichment (geo-IP, threat intelligence lookup)50 - Design search and query capabilities for incident investigation51524. **Define retention policy:**53 - **Security logs** — 1 year minimum (authentication, authorization, access control events)54 - **Audit logs** — 7 years for compliance (SOX, PCI DSS, HIPAA as applicable)55 - **Application logs** — 90 days hot storage, 1 year cold storage56 - **Infrastructure logs** — 30 days hot storage, 6 months cold storage57 - Define storage tiers (hot/warm/cold/archive) with cost optimization58 - Set automated lifecycle policies for tier transitions and deletion59605. **Protect log integrity:**61 - Store logs in immutable storage (write-once, read-many) for audit trail62 - Implement checksums or digital signatures for log entries63 - Restrict access to log data (separate from application access controls)64 - Ensure log deletion requires multi-party approval65 - Design tamper-evident log chains for high-security requirements66676. **Define access control for log data:**68 - Restrict log access by role (security team: full access, developers: own service logs, auditors: read-only)69 - Implement PII masking in log views for non-privileged users70 - Audit log access itself (who searched what, when)71 - Define self-service log access for developers within guardrails72737. **Write the artifact** — append to `.singularskills/security/<project>/03-detect.md` under a `## Logging Architecture` section. Create the file with a `# Detect — <project>` heading if it does not exist; if a `## Logging Architecture` section already exists, replace only that section. The section contains:7475 - **Log Sources** — all sources with type, volume estimate, and sensitivity76 - **Log Format Standard** — mandatory fields, correlation ID strategy, and sanitization rules77 - **Aggregation Architecture** — platform selection, collection design, and capacity plan78 - **Retention Policy** — retention periods by log type with storage tier strategy79 - **Log Integrity** — immutability, checksums, and tamper protection measures80 - **Access Control** — role-based log access and PII masking approach8182## Output8384The logging architecture appended to `.singularskills/security/<project>/03-detect.md`. Present a summary to the user highlighting:85- Log sources to be aggregated and estimated volume86- Chosen aggregation platform and architecture87- Retention periods by log type and compliance alignment88- Log integrity and access control measures8990**Record the run.** After you have written your section, append this line to `.singularskills/log.jsonl` (the `started` line of Prerequisites, with `"event":"finished"`): `{"ts":"<ISO timestamp>","skill":"security-logging-architecture","workflow":"security","phase":3,"project":"<project>","event":"finished","artifact":"03-detect.md","note":""}`.9192This artifact is shared by every skill of phase 3 (Detect). The frontmatter is written once, by whoever creates the file: if you create it, open it with the block below (`skill` is the phase title, `started` the time of that first write; `gate` is `passed` when the gate ran clean and `skipped` under `--skip-checks`); if it already exists, leave the block alone except to update `finished` and to set `gate` to the worst value among the sections (`skipped` over `passed`).9394```yaml95---96skill: Detect97workflow: security98phase: 399project: <project>100started: <ISO timestamp>101finished: <ISO timestamp>102agent: <the agent you run in, or "unknown">103gate: <passed|skipped>104---105```106107Open your `## Logging Architecture` section with one italic line, `*<skill name> · <ISO timestamp> · gate: <passed|skipped>*` (your skill name is `security-logging-architecture`).108109End your section with `### Decisions` and `### Open questions` (one level under your `## Logging Architecture` heading, so the section stays one block), in this shape:110111```markdown112### Decisions113- <what was chosen> — <why> — <what was rejected>114115### Open questions116- <what the next phase or a reviewer must settle>117```118119End the artifact with this paragraph, verbatim:120121> **Not a penetration test or audit.** This artifact is generated analysis; validate findings against the live system and have material controls reviewed by a qualified security professional.