Azure Monitor & Log Analytics for Security
Microsoft Sentinel runs on a Log Analytics workspace. Defender for Cloud, Defender XDR
hunting tables, Activity logs, and Diagnostic settings also flow there. The workspace
strategy decides whether your security telemetry is complete, compliant, and
affordable — or two of three.
This skill covers the security-side decisions in workspace design: tiering, retention,
DCRs, RBAC, and cost.
When to use
Designing or refactoring the Log Analytics workspace(s) that back Sentinel and the
broader security telemetry estate.
Do not use this skill for Sentinel detection authoring
(sentinel-detection-engineering), App Insights instrumentation
(appinsights-instrumentation), or non-security observability.
Log table tiers — pick per table
| Tier |
Query latency |
Retention max |
Use case |
| Analytics |
Sub-second |
730 days interactive |
Detections, hunting, dashboards |
| Basic |
Seconds (KQL with limits) |
30 days interactive + archive |
High-volume "search if needed" logs |
| Auxiliary |
Slower / async |
12 months interactive |
Low-touch compliance archive feeds |
| Archive (any tier) |
Search-job recall |
up to 12 years |
Long-tail evidence, regulatory hold |
Microsoft has been re-naming/re-shaping these — verify current naming and pricing in your
region before commit.
Approach
Workspace topology.
- Default: one Sentinel workspace per tenant for security signal — keeps
correlation easy.
- Regional / sovereign: separate workspaces where data residency or sovereignty
requires.
- Multi-workspace Sentinel: only when business reasons (M&A boundary, MSSP
model) force it. Cross-workspace queries work but add complexity.
- Don't split per-team; that fragments correlation.
Use Azure Monitor Agent (AMA) for everything new. The Microsoft Monitoring Agent
(MMA / Log Analytics agent) is retired. Migration paths via Defender for Cloud
auto-provisioning and Sentinel solutions.
Define DCRs per data class.
- One DCR per data source family (Windows security events, Linux Syslog, custom
logs).
- Use transformKql to drop noisy columns or whole event IDs at ingest. A 30%
pre-ingest reduction beats any post-ingest tuning.
- Validate transformations against detection KQL — don't drop a column a Sentinel
analytic relies on.
Tier per table.
- Analytics: SecurityEvent (filtered), SigninLogs, AuditLogs, DeviceLogonEvents
(Defender XDR), Sentinel-driven detection sources.
- Basic: AzureDiagnostics for chatty resources, NSG flow logs, web server access
logs (where you need them only for incident search).
- Auxiliary: long-retention compliance feeds (e.g., raw firewall logs for 12
months).
Tier choice changes ingestion + retention costs significantly.
Retention and archive.
- Interactive retention: 90 days default — extend to align with detection lookback
windows (often 180–730 days).
- Archive (cold) for compliance and IR replay (12-year max for some scenarios).
- Search jobs rehydrate archived data when needed.
Cost control.
- Commitment tier at the workspace (100 GB/day → 5,000 GB/day) cuts per-GB
price ~15–30%.
- Identify and cap top sources (DNS, NSG flow, Linux Syslog) — usually 80% of cost.
- Daily cap as a guardrail (drops data above cap; alert on hit).
- Review monthly: top tables, top resources, ingest trend.
Customer-managed keys (CMK) on the workspace if your data classification
requires. Plan for the dedicated cluster requirement and the operational complexity.
RBAC.
- Workspace-level: Log Analytics Reader, Log Analytics Contributor.
- Table-level RBAC: scope analyst access to specific tables (e.g., HR mailbox
audit visible only to insider-risk team).
- Sentinel: separate Sentinel Reader / Responder / Contributor roles.
- Avoid Owner on the workspace for SOC analysts.
Cross-workspace and cross-tenant.
- Sentinel workspace manager (preview/GA per region) for MSSP-style multi-tenant.
- Cross-workspace queries via union for federated estates.
Guardrails
- Default 90-day retention is rarely enough for security investigations. APT
dwell-time medians exceed that. Plan 180–730 days for primary detection sources.
- Don't ingest everything in Analytics tier "to be safe." Cost spirals; add Basic
for "store and search-if-needed."
- Transformations are powerful and dangerous. A column drop today breaks a
detection tomorrow. Source-control DCRs and review changes.
- Daily cap as a cost control drops data — including security data. Pair with
alerting; never set a cap below high-water mark for security tables.
- Don't deploy MMA in 2025+. Use AMA. Existing MMA workloads need a migration plan.
- CMK on workspace requires dedicated cluster and ≥500 GB/day commitment. Big
operational and cost step.
- Multi-workspace Sentinel = harder hunting. Justify per business reason, not per
team preference.
- Archive recall is search jobs, not native query. SLA assumptions for IR change.
- Diagnostic settings are per resource; per-subscription policy is what scales.
Enforce via Azure Policy.
Common anti-patterns
- "Per-team workspace because each team wants 'their data'" — correlation breaks,
cost balloons.
- "Analytics tier for everything" — 5x cost vs Basic for tables you query
reactively.
- "NSG flow logs full-fidelity to Sentinel forever" — top cost driver. Sample,
transform, or Basic-tier.
- "30-day retention to keep cost down" — incidents discovered in week 6 have no
data.
- "Daily cap below normal volume" — silent data loss, missed detections.
- "DCRs hand-edited in the portal across 50 subs" — drift, no audit. Bicep + Azure
Policy.
- "Sentinel + Defender XDR in two separate workspaces" — cross-product hunting
pain. Unify.
- "Upgraded to dedicated cluster for CMK without volume justification" — cost
surprise.
Example prompts
Design the Log Analytics workspace strategy for a 100,000-employee multi-region Sentinel deployment with sovereignty constraints in EU and APAC.
Migrate 6,000 servers from MMA to AMA via Defender for Cloud auto-provisioning; identify gaps in detection coverage during cutover.
Author DCRs that drop high-noise columns from SecurityEvent and Syslog with 35% expected ingest reduction.
Tier plan: which tables go Analytics vs Basic vs Auxiliary for a Sentinel + DfC workload, with retention.
Cost optimization: identify top 10 cost drivers in our 12 TB/day workspace and propose targeted reductions.
Configure table-level RBAC so insider-risk analysts see EmailEvents but not DeviceProcessEvents.
Customer-managed key rollout for the security workspace — dedicated cluster, key rotation, recovery runbook.
Daily cap and ingestion alerting strategy that protects budget without dropping detection-critical tables.
Microsoft Learn
1---2name: azure-monitor-security3description: "Guidance for the security-side use of Azure Monitor and Log Analytics — designing the workspace strategy that feeds Microsoft Sentinel and Defender for Cloud, choosing analytics vs basic vs auxiliary log tiers, retention and archive, table-level transformations to drop noise pre-ingestion, Data Collection Rules (DCRs) and Azure Monitor Agent (AMA), workspace topology (single vs regional vs sovereign), commitment tiers and cost control, customer-managed keys for the workspace, RBAC including table-level RBAC, integration with Sentinel (workspace requirements), and decommissioning legacy MMA. WHEN: Log Analytics workspace design, Azure Monitor security data, Sentinel workspace, log tier basic auxiliary, AMA Azure Monitor Agent, DCR data collection rule, drop columns ingestion transform, log retention archive, workspace CMK, table-level RBAC, log ingestion cost control. DO NOT USE for Sentinel detection authoring (use sentinel-detection-engineering), App Insights instrumentation (use appinsights-instrumentation), or non-security observability."4license: MIT5metadata:6 source: https://skillmd.com/skills/vinayaklatthe/azure-monitor-security7 author: Microsoft8 version: "0.1.0"9---1011# Azure Monitor & Log Analytics for Security1213Microsoft Sentinel runs on a Log Analytics workspace. Defender for Cloud, Defender XDR14hunting tables, Activity logs, and Diagnostic settings also flow there. The workspace15strategy decides whether your security telemetry is **complete, compliant, and16affordable** — or two of three.1718This skill covers the security-side decisions in workspace design: tiering, retention,19DCRs, RBAC, and cost.2021## When to use22Designing or refactoring the Log Analytics workspace(s) that back Sentinel and the23broader security telemetry estate.2425**Do not use this skill** for Sentinel detection authoring26(`sentinel-detection-engineering`), App Insights instrumentation27(`appinsights-instrumentation`), or non-security observability.2829## Log table tiers — pick per table3031| Tier | Query latency | Retention max | Use case |32|---|---|---|---|33| **Analytics** | Sub-second | 730 days interactive | Detections, hunting, dashboards |34| **Basic** | Seconds (KQL with limits) | 30 days interactive + archive | High-volume "search if needed" logs |35| **Auxiliary** | Slower / async | 12 months interactive | Low-touch compliance archive feeds |36| **Archive** (any tier) | Search-job recall | up to 12 years | Long-tail evidence, regulatory hold |3738Microsoft has been re-naming/re-shaping these — verify current naming and pricing in your39region before commit.4041## Approach42431. **Workspace topology.**44 - **Default**: one Sentinel workspace per tenant for security signal — keeps45 correlation easy.46 - **Regional / sovereign**: separate workspaces where data residency or sovereignty47 requires.48 - **Multi-workspace Sentinel**: only when business reasons (M&A boundary, MSSP49 model) force it. Cross-workspace queries work but add complexity.50 - Don't split per-team; that fragments correlation.51522. **Use Azure Monitor Agent (AMA) for everything new.** The Microsoft Monitoring Agent53 (MMA / Log Analytics agent) is **retired**. Migration paths via Defender for Cloud54 auto-provisioning and Sentinel solutions.55563. **Define DCRs per data class.**57 - One DCR per data source family (Windows security events, Linux Syslog, custom58 logs).59 - Use **transformKql** to drop noisy columns or whole event IDs at ingest. A 30%60 pre-ingest reduction beats any post-ingest tuning.61 - Validate transformations against detection KQL — don't drop a column a Sentinel62 analytic relies on.63644. **Tier per table.**65 - **Analytics**: SecurityEvent (filtered), SigninLogs, AuditLogs, DeviceLogonEvents66 (Defender XDR), Sentinel-driven detection sources.67 - **Basic**: AzureDiagnostics for chatty resources, NSG flow logs, web server access68 logs (where you need them only for incident search).69 - **Auxiliary**: long-retention compliance feeds (e.g., raw firewall logs for 1270 months).71 Tier choice changes ingestion + retention costs significantly.72735. **Retention and archive.**74 - Interactive retention: 90 days default — extend to align with detection lookback75 windows (often 180–730 days).76 - Archive (cold) for compliance and IR replay (12-year max for some scenarios).77 - **Search jobs** rehydrate archived data when needed.78796. **Cost control.**80 - **Commitment tier** at the workspace (100 GB/day → 5,000 GB/day) cuts per-GB81 price ~15–30%.82 - Identify and cap top sources (DNS, NSG flow, Linux Syslog) — usually 80% of cost.83 - **Daily cap** as a guardrail (drops data above cap; alert on hit).84 - Review monthly: top tables, top resources, ingest trend.85867. **Customer-managed keys (CMK)** on the workspace if your data classification87 requires. Plan for the dedicated cluster requirement and the operational complexity.88898. **RBAC.**90 - **Workspace-level**: Log Analytics Reader, Log Analytics Contributor.91 - **Table-level RBAC**: scope analyst access to specific tables (e.g., HR mailbox92 audit visible only to insider-risk team).93 - Sentinel: separate Sentinel Reader / Responder / Contributor roles.94 - Avoid Owner on the workspace for SOC analysts.95969. **Cross-workspace and cross-tenant.**97 - **Sentinel workspace manager** (preview/GA per region) for MSSP-style multi-tenant.98 - **Cross-workspace queries** via union for federated estates.99100## Guardrails101- **Default 90-day retention is rarely enough for security investigations.** APT102 dwell-time medians exceed that. Plan 180–730 days for primary detection sources.103- **Don't ingest everything in Analytics tier "to be safe."** Cost spirals; add Basic104 for "store and search-if-needed."105- **Transformations are powerful and dangerous.** A column drop today breaks a106 detection tomorrow. Source-control DCRs and review changes.107- **Daily cap as a cost control drops data — including security data.** Pair with108 alerting; never set a cap below high-water mark for security tables.109- **Don't deploy MMA in 2025+.** Use AMA. Existing MMA workloads need a migration plan.110- **CMK on workspace requires dedicated cluster and ≥500 GB/day commitment.** Big111 operational and cost step.112- **Multi-workspace Sentinel = harder hunting.** Justify per business reason, not per113 team preference.114- **Archive recall is search jobs, not native query.** SLA assumptions for IR change.115- **Diagnostic settings are per resource; per-subscription policy is what scales.**116 Enforce via Azure Policy.117118## Common anti-patterns119- **"Per-team workspace because each team wants 'their data'"** — correlation breaks,120 cost balloons.121- **"Analytics tier for everything"** — 5x cost vs Basic for tables you query122 reactively.123- **"NSG flow logs full-fidelity to Sentinel forever"** — top cost driver. Sample,124 transform, or Basic-tier.125- **"30-day retention to keep cost down"** — incidents discovered in week 6 have no126 data.127- **"Daily cap below normal volume"** — silent data loss, missed detections.128- **"DCRs hand-edited in the portal across 50 subs"** — drift, no audit. Bicep + Azure129 Policy.130- **"Sentinel + Defender XDR in two separate workspaces"** — cross-product hunting131 pain. Unify.132- **"Upgraded to dedicated cluster for CMK without volume justification"** — cost133 surprise.134135## Example prompts136- `Design the Log Analytics workspace strategy for a 100,000-employee multi-region137 Sentinel deployment with sovereignty constraints in EU and APAC.`138- `Migrate 6,000 servers from MMA to AMA via Defender for Cloud auto-provisioning;139 identify gaps in detection coverage during cutover.`140- `Author DCRs that drop high-noise columns from SecurityEvent and Syslog with 35%141 expected ingest reduction.`142- `Tier plan: which tables go Analytics vs Basic vs Auxiliary for a Sentinel + DfC143 workload, with retention.`144- `Cost optimization: identify top 10 cost drivers in our 12 TB/day workspace and145 propose targeted reductions.`146- `Configure table-level RBAC so insider-risk analysts see EmailEvents but not147 DeviceProcessEvents.`148- `Customer-managed key rollout for the security workspace — dedicated cluster, key149 rotation, recovery runbook.`150- `Daily cap and ingestion alerting strategy that protects budget without dropping151 detection-critical tables.`152153## Microsoft Learn154- Log Analytics workspace design: https://learn.microsoft.com/azure/azure-monitor/logs/workspace-design155- Table tiers (Analytics / Basic / Auxiliary): https://learn.microsoft.com/azure/azure-monitor/logs/data-platform-logs#table-plans156- Retention and archive: https://learn.microsoft.com/azure/azure-monitor/logs/data-retention-configure157- Azure Monitor Agent: https://learn.microsoft.com/azure/azure-monitor/agents/azure-monitor-agent-overview158- Data collection rules: https://learn.microsoft.com/azure/azure-monitor/data-collection/data-collection-rule-overview159- Ingestion-time transformations: https://learn.microsoft.com/azure/azure-monitor/data-collection/data-collection-transformations160- Commitment tiers and pricing: https://learn.microsoft.com/azure/azure-monitor/logs/cost-logs161- Workspace customer-managed keys: https://learn.microsoft.com/azure/azure-monitor/logs/customer-managed-keys162- Table-level RBAC: https://learn.microsoft.com/azure/azure-monitor/logs/manage-access#table-level-azure-rbac163- Sentinel + workspace requirements: https://learn.microsoft.com/azure/sentinel/prerequisites164- MMA retirement guidance: https://learn.microsoft.com/azure/azure-monitor/agents/azure-monitor-agent-migration