SQL Server ERRORLOG Review Skill
Purpose
Parse and analyze SQL Server ERRORLOG content to surface operational warnings, high-availability
failures, resource pressure signals, security events, and configuration anomalies. Applies 33
checks (E1–E33) across six categories:
- E1–E8 — AG / High Availability: failovers, lease expiry, replica state changes, synchronization errors
- E9–E14 — Memory and resource pressure: page allocation failures, OS paging, worker exhaustion, non-yielding schedulers
- E15–E19 — I/O and storage: slow I/O subsystem, corruption warnings, tempdb exhaustion, log backup gaps, VLF proliferation
- E20–E24 — Startup, shutdown, and connectivity: abnormal termination, restart cycling, login failure bursts, linked server errors
- E25–E28 — Configuration and informational: trace flags, unconfigured max memory, log rotation gaps, version end-of-support
- E29–E33 — SQL 2019/2022 modern features: ADR PVS cleanup stall, IQP DOP feedback, Ledger verification failure, CE feedback model change, Azure Arc agent disconnect
Input
Accept any of:
- File path — path to the SQL Server ERRORLOG file (default location:
C:\Program Files\Microsoft SQL Server\MSSQL<ver>.<inst>\MSSQL\Log\ERRORLOG)
- Inline paste — raw ERRORLOG text pasted directly into chat; partial excerpts are valid
- Natural language description — describe the symptoms or paste selected log lines with context
For best results, provide the current ERRORLOG and at least one prior log (ERRORLOG.1). When
only partial content is available, state which time range is covered.
Capture via T-SQL
-- Read current ERRORLOG (0 = current, 1 = previous, 2 = the one before that)
EXEC xp_readerrorlog 0, 1; -- SQL Server log, current file
EXEC xp_readerrorlog 1, 1; -- SQL Server log, previous file
-- Filter to AG-related messages only
EXEC xp_readerrorlog 0, 1, N'availability', NULL, NULL, NULL, N'desc';
-- Filter to a time window (last 2 hours)
DECLARE @start DATETIME = DATEADD(HOUR, -2, GETDATE());
EXEC xp_readerrorlog 0, 1, NULL, NULL, @start, NULL, N'desc';
Column Reference
| Column |
Meaning |
| LogDate |
Timestamp of the log entry (datetime2 precision) |
| ProcessInfo |
SPID or system process (e.g., spid28s, Logon, Backup) |
| Text |
Log message text |
Thresholds Reference
| Threshold |
Value |
Used by |
| Login failure burst — Warning |
> 5 Login failed messages in any 5-min window |
E22 |
| Login failure burst — Critical |
> 20 Login failed messages in any 5-min window |
E22 |
| Restart cycling |
≥ 2 SQL Server startup messages within 60 min |
E21 |
| I/O slow built-in threshold |
15 seconds (SQL Server internal, non-configurable) |
E15 |
| Log backup overdue — FULL/BULK_LOGGED |
> 24 hr since last Database backed up entry |
E18 |
| Log backup overdue — active log pressure signal |
> 8 hr when log_reuse_wait_desc = LOG_BACKUP |
E18 |
AG / High Availability Checks (E1–E8)
E1 — AG Failover Event
- Trigger: Log contains
is changing roles from or is preparing to transition to or automatic failover
in the same entry or within the same minute as a role-change message; also in response to a request from the Windows Server Failover Cluster
- Severity: Warning — planned failover expected; Critical if the word
automatic appears
(unplanned loss of primary)
- Fix: For unplanned failovers, check E2 (lease expiry) and E6 (health check timeout) as
probable root causes. For planned failovers in unexpected windows, review change-management
records. Run
/sqlwait-review on HADR_SYNC_COMMIT and HADR_WORK_QUEUE waits.
E2 — Lease Expiry
- Trigger: Log contains
lease between the availability group and the Windows Server Failover Cluster has expired or The lease of availability group combined with has expired
- Severity: Critical — lease expiry is the most common root cause of unplanned AG failovers
- Fix: Investigate the time immediately before this entry for E15 (slow I/O), E13
(non-yielding scheduler), or OS-level events. Common causes: storage latency spike causing
the sp_server_diagnostics thread to miss its deadline, high CPU starvation, or WSFC network
interruption. Increase
LeaseTimeout in WSFC only as a temporary measure — fix the root cause.
E3 — Replica State Change
- Trigger: Log contains
The local replica of availability group ... is changing roles or
is preparing to transition to the
- Severity: Warning — state transitions are normal during planned operations; unexpected
transitions during business hours warrant investigation
- Fix: Correlate the timestamp with E1 (failover), E2 (lease), or external WSFC events.
If unplanned, check the Windows Event Log and WSFC cluster log for the triggering event.
E4 — AG Database Joining Failure
- Trigger: Log contains
Failed to join local availability database or The availability database ... is not in the correct state
- Severity: Critical — the AG database is not receiving redo; secondary is running but not
synchronized, providing false HA coverage
- Fix: Run
SELECT * FROM sys.dm_hadr_database_replica_states to check
synchronization_state_desc and redo_queue_size. If redo queue is growing, check disk I/O
on the secondary. If the database is in NOT SYNCHRONIZING, re-join: ALTER DATABASE [db] SET HADR AVAILABILITY GROUP = [ag_name].
E5 — Data Synchronisation Suspended
- Trigger: Log contains
Synchronization of this database ... has been suspended or
Data movement for availability database ... has been suspended
- Severity: Warning — a suspended database is not receiving log records; RPO clock is running
- Fix: Identify whether the suspension was manual (
ALTER DATABASE ... SET HADR SUSPEND) or
automatic (error-triggered). Check for E15/E16 (I/O or corruption) causing automatic suspension.
Resume: ALTER DATABASE [db] SET HADR RESUME. Monitor redo queue.
E6 — AG Health Check Timeout
- Trigger: Log contains
availability group ... has failed to take necessary action within the time allotted or The availability group ... exceeded the health-check timeout
- Severity: Critical — health-check failure directly precedes automatic failover; this entry
combined with E1 confirms the full failover sequence
- Fix: Identify what the primary was doing at the time. E13 (non-yielding scheduler) or E9
(page allocation failure) are common co-occurrences. The
HealthCheckTimeout WSFC property
controls sensitivity — do not increase it without fixing the underlying responsiveness problem.
E7 — Redo Thread Error
- Trigger: Log contains
An error occurred in the redo thread for database or
Redo thread for database ... encountered error
- Severity: Critical — the secondary redo thread has failed; the secondary is no longer
applying log records and RPO is accumulating
- Fix: Note the error number in the log message. Common causes: corruption on the secondary
(check E16), log record version mismatch after an upgrade, or disk full on secondary. For
disk-full, free space and resume synchronization. For corruption, restore the secondary from
a backup and re-seed.
E8 — Secondary Not Synchronising
- Trigger: Log contains
Waiting for redo catch-up or mentions secondary redo queue in a
warning context; or log send queue appearing repeatedly with growing values
- Severity: Warning — secondary is lagging; failover to this replica would result in data
loss proportional to the redo queue depth
- Fix: Check network bandwidth between primary and secondary. Run
SELECT redo_queue_size, redo_rate FROM sys.dm_hadr_database_replica_states. If redo rate <
log generation rate, the secondary cannot keep up — review disk I/O on secondary (E15) or
increase network bandwidth.
Memory and Resource Pressure Checks (E9–E14)
E9 — FAIL_PAGE_ALLOCATION
- Trigger: Log contains
FAIL_PAGE_ALLOCATION (exact string, case-insensitive)
- Severity: Critical — SQL Server could not satisfy an internal memory allocation; queries
may have failed with out-of-memory errors; this entry often precedes OS paging (E10)
- Fix: Check
max server memory configuration (E26). Run
SELECT type, pages_kb FROM sys.dm_os_memory_clerks ORDER BY pages_kb DESC to identify
which clerk is consuming the most memory. Consider reducing max server memory by 10–15% to
leave headroom for OS and other processes.
E10 — OS Memory Pressure
- Trigger: Log contains
A significant part of sql server process memory has been paged out
or Working set trim
- Severity: Critical — Windows has paged SQL Server memory to disk under OS memory pressure;
buffer pool pages are on disk, causing extreme I/O latency
- Fix: Reduce
max server memory to allow OS headroom (leave at least 10% of RAM or 4 GB,
whichever is greater). Enable Lock Pages in Memory (LPIM) to prevent paging for 64-bit SQL
Server service account. Investigate other processes competing for RAM on the host.
E11 — Memory Insufficient (Resource Pool or Buffer Pool)
- Trigger: Log contains
There is insufficient memory available in the buffer pool (error 802, buffer pool full) OR There is insufficient system memory in resource pool (error 701, Resource Governor pool exhausted) OR Memory Manager: Memory node available memory is less than threshold
- Severity: Critical — queries requiring memory grants are being denied; workload will stall on
RESOURCE_SEMAPHORE waits
- Fix: Run
/sqlwait-review and check for RESOURCE_SEMAPHORE dominance. Increase max server memory if physical RAM allows, or reduce min memory per query via Resource Governor. Identify large-grant queries with /sqlplan-review S2–S4. Error 802 (buffer pool) and error 701 (resource pool) require different fixes: 802 → increase max server memory or reduce buffer pool competition; 701 → adjust Resource Governor pool memory limits.
E12 — Worker Thread Exhaustion
- Trigger: Log contains
There are no more threads available to process new requests or
Worker Thread ... has been waiting too long
- Severity: Critical — new connections are being refused or queued; the instance is at
maximum worker thread capacity
- Fix: Increase
max worker threads via sp_configure only after identifying root cause.
Common causes: blocking chains holding threads (check sys.dm_exec_requests), long-running
queries, or undersized max worker threads for the workload. Run /sqlwait-review for
THREADPOOL waits (V-checks).
E13 — Scheduler Non-Yielding
- Trigger: Log contains
Process appears to be non-yielding on Scheduler or
A scheduler appears to be non-yielding
- Severity: Critical — a thread is monopolising a scheduler without yielding; this blocks
all other threads on that scheduler, degrades responsiveness, and can trigger AG health-check
timeouts (E6) and lease expiry (E2)
- Fix: A memory dump is typically generated automatically. Look for a
.mdmp file in the
SQL Server Log directory matching the timestamp. Common causes: large in-memory sort, CLR
call, XTP operation, or a bug in a specific build — check if a known hotfix applies for the
version (E28). Consider enabling DBCC TRACEON(8086) on advice from Microsoft Support.
E14 — Memory Grant Timeout
- Trigger: Log contains
Memory grant request timed out or
A request for memory failed with OOM (out of memory) status
- Severity: Warning — a query could not acquire its requested memory grant within the
timeout; it may have been killed or retried with a reduced grant, causing a spill to TempDb
- Fix: Capture the affected query and run
/sqlplan-review for S2–S4 (memory grant checks).
Update statistics to improve cardinality estimates. Use Resource Governor to cap grants for
ad-hoc workloads. Check for E11 (resource pool exhaustion) as a co-trigger.
I/O and Storage Checks (E15–E19)
E15 — I/O Subsystem Slow
- Trigger: Log contains
SQL Server has encountered combined with I/O requests taking longer than 15 seconds (SQL Server's built-in slow I/O threshold)
- Severity: Critical — storage latency has exceeded the 15-second internal threshold;
this is a primary trigger for AG lease expiry (E2) and health-check timeouts (E6)
- Fix: Note the file path and database in the message. Investigate storage subsystem: check
disk queue length, RAID controller cache status, SAN/NVMe latency metrics, and any concurrent
backup or maintenance operations competing for I/O. If on a VM, check storage IOPS limits.
Run
/sqlwait-review for PAGEIOLATCH_SH and PAGEIOLATCH_EX dominance.
E16 — Database Corruption Warning
- Trigger: Log contains
checksum mismatch, torn page, consistency errors detected,
DBCC CHECKDB found with error counts > 0, Error: 823 (OS-level I/O failure — Windows API
returned an error), Error: 824 (logical consistency check failure — Windows I/O succeeded but
SQL detected corruption on the page), or Error: 825 (read succeeded after retry — transient
storage issue; Warning severity; indicates potential hardware degradation)
- Severity: Critical for Msg 823 and Msg 824 (data corruption confirmed); Warning for Msg 825
(read ultimately succeeded but is a precursor to harder failures — investigate storage hardware
immediately); Critical if DBCC CHECKDB reports allocation or consistency errors
- Fix: Run
DBCC CHECKDB ([database]) WITH NO_INFOMSGS immediately to assess scope. Do
not attempt to repair until a current, verified backup exists. For REPAIR_ALLOW_DATA_LOSS,
treat it as a last resort — restore from backup is always preferable. Investigate E15 (I/O
latency) and storage hardware health as root causes.
E17 — TempDB Space Exhaustion
- Trigger: Log contains
Could not allocate space combined with in database 'tempdb'
or tempdb is full or tempdb ran out of space
- Severity: Critical — queries requiring temporary space (sorts, hashes, spools, row
versioning) are failing; error 1105 is returned to applications
- Fix: Immediately:
DBCC SHRINKFILE on tempdb data files to recover any unused allocated
space, or add a tempdb data file. Long term: investigate which query is consuming tempdb
(check sys.dm_db_session_space_usage). Run /sqlplan-review for N41–N43 (spill operators).
Consider pre-allocating tempdb to expected working size at startup.
E18 — Log Backup Overdue
- Trigger: Gap between consecutive
Database backed up entries for the same database
exceeds the threshold for that recovery model. For databases in FULL or BULK_LOGGED recovery,
flag if the gap exceeds 24 hours; flag more urgently if log backup entries are absent while
other evidence suggests active transaction log growth
- Severity: Warning — log space will grow unboundedly without log backups; in a FULL
recovery database, the log cannot be truncated until backed up
- Fix: Run a log backup immediately:
BACKUP LOG [database] TO DISK = N'path\logbackup.bak'.
Verify the SQL Agent log backup job is scheduled and enabled. Check sys.databases column
log_reuse_wait_desc — if LOG_BACKUP, the log is waiting for a backup to allow truncation.
E19 — VLF Proliferation Signal
- Trigger: Log shows repeated
autogrow events on transaction log files (multiple autogrow
completions in the log window), or the database log has grown significantly between ERRORLOG
entries — inferred from repeated log file path growth messages
- Severity: Info — excessive VLFs degrade recovery time and log-backup performance; auto-grow
events indicate the log was not sized for the workload
- Fix: Shrink and pre-size the log: set the initial log file size to cover expected working
set and disable autogrow on the log (or set a large, infrequent growth increment). Run
DBCC LOGINFO ([database]) to count current VLFs — if > 1,000, shrink and re-expand in one
step. Align with E18 (log backup cadence) to ensure the log truncates regularly.
Startup, Shutdown, and Connectivity Checks (E20–E24)
E20 — Abnormal Shutdown
- Trigger: Log contains
SQL Server is terminating or SQL Server has encountered combined
with stack dump or shutdown messages, without a preceding graceful shutdown marker
(SQL Server is terminating due to a system shutdown request at the end of the prior log file)
- Severity: Critical — the instance crashed rather than shut down cleanly; uncommitted
transactions were rolled back on restart; any in-flight work is lost
- Fix: Check the Windows Event Log (
Application and System sources) for the crash
timestamp. Look for a dump file in the SQL Server Log directory. If the crash occurred
mid-transaction in an AG, check whether secondary databases advanced beyond the primary
(split-brain risk). Engage Microsoft Support with the minidump if the crash is reproducible.
E21 — Repeated Restarts
- Trigger: ERRORLOG or combined ERRORLOG + ERRORLOG.1 contains ≥ 2 SQL Server startup
messages (lines containing
SQL Server is starting or This instance of SQL Server last reported using a process ID) within a 60-minute window
- Severity: Critical — the instance is crash-looping; each restart drops all plan cache and
connection state; applications experience repeated connection failures
- Fix: Check E20 (abnormal shutdown) for the crash cause between restarts. If the instance
is restarting due to a failed startup condition (e.g., tempdb creation failure, master database
corruption, or xp_cmdshell misconfiguration), resolve the startup error first. Enable Windows
Automatic Recovery only after identifying the underlying fault.
E22 — Login Failure Burst
- Trigger: Count of
Login failed entries exceeds the threshold within a 5-minute rolling
window — see Thresholds Reference for Warning and Critical levels
- Severity: Warning if > 5 failures in 5 min; Critical if > 20 failures in 5 min
- Fix: Identify the
ClientConnectionID and source IP in the failure messages. A burst from
one account likely indicates a misconfigured application connection string after a password
rotation. A burst from many accounts may indicate a brute-force attempt. For brute-force:
enable SQL Server Audit or Extended Events on Failed Logins and block the source IP at the
network layer. Ensure LOGINAUDIT is set to Failed logins only or Both in Server
properties so future bursts appear in the ERRORLOG.
E23 — Linked Server Error
- Trigger: Log contains
OLE DB provider combined with reported an error or
Cannot obtain the required interface for a linked server provider
- Severity: Warning — distributed queries or cross-server stored procedures using this
linked server will fail until the provider error is resolved
- Fix: Identify the linked server name and provider from the error text. Common causes:
target server unavailable, credential expiry, or OLE DB provider version mismatch. Test
connectivity:
EXEC sp_testlinkedserver [linked_server_name]. If the provider is outdated,
update it on the SQL Server host.
E24 — Connectivity Error
- Trigger: Log contains
A connection was successfully established with the server, but then an error occurred during the login process or The connection has been lost or
A network-related or instance-specific error in the ERRORLOG (as opposed to the client)
- Severity: Warning — SQL Server is logging errors from its own outbound connections
(linked servers, distributed queries, SSISDB, mail, replication) or from incoming connections
that dropped after TCP establishment
- Fix: Correlate the timestamp with E22 (login failures), network infrastructure changes,
or TLS/SSL certificate renewals. If
TLS handshake appears in the message, verify that
the certificate in use has not expired and that the client supports the negotiated protocol.
Configuration and Informational Checks (E25–E28)
E25 — Trace Flag Active
- Trigger: Log contains
Trace flag combined with is set or was enabled at startup
in startup messages
- Severity: Info — trace flags change engine behavior; document intent and verify they
are still appropriate for the current SQL Server version
- Fix: List all active trace flags:
DBCC TRACESTATUS(-1). Common production trace flags
and their intent: 1117/1118 (tempdb allocation — superseded in 2016+), 3226 (suppress
successful backup log entries), 4199 (QO hotfixes). Remove trace flags that are no longer
needed or that apply to behaviour fixed in a later CU.
E26 — Max Server Memory Default
- Trigger: Log contains startup line showing
max server memory = 2147483647 MB, or the
instance has been running with the default (unlimited) memory configuration — inferred from
startup messages or the absence of an explicit max server memory setting entry
- Severity: Info — unlimited memory allows SQL Server to consume all available RAM, starving
the OS and any other services, which can trigger E10 (OS paging)
- Fix: Set
max server memory to total RAM minus OS headroom: leave at least 10% of RAM or
4 GB (whichever is larger) for the OS. For example, on a 64 GB server:
EXEC sp_configure 'max server memory (MB)', 57344; RECONFIGURE;
E27 — ERRORLOG Rotation Gap
- Trigger: Only a single ERRORLOG file is provided, covering a window shorter than 24 hours,
with no prior context from ERRORLOG.1 or earlier
- Severity: Info — events before the current file (including the original startup, prior
AG failovers, or earlier memory events) are not visible; findings may be incomplete
- Fix: Retrieve prior ERRORLOG files:
EXEC xp_readerrorlog 1, 1 through
EXEC xp_readerrorlog 6, 1 (SQL Server retains up to 6 prior logs by default, configurable
in SSMS → Server Properties → Database Settings → Number of error log files). State in the
report: "Analysis covers [start] – [end] only; prior events not available."
E28 — SQL Server Version
- Trigger: Startup line containing
Microsoft SQL Server 20XX version string — present
in every ERRORLOG at instance start
- Severity: Info — extract and evaluate: (1) is this build on extended support, mainstream
support, or past end-of-support? (2) is this the latest CU for this major version?
- Fix: Compare the build number in the log against the Microsoft SQL Server build list.
If past end-of-support (e.g., SQL 2014 Extended Support ended 2024-07-09 — 2019 was only its
mainstream support end; SQL 2016 Extended Support ends 2026-07-14), plan
upgrade. If not on the latest CU, evaluate whether open bugs fixed in later CUs are relevant
to the observed issues. Report the version string verbatim in the Output Summary.
SQL 2019/2022 Modern Feature Checks (E29–E33)
E29 — ADR PVS Cleanup Stall
- Trigger: Log contains
Persistent Version Store cleanup with stall or unable to advance — SQL 2019+; skip if compat level < 150
- Severity: Warning — ADR (Accelerated Database Recovery) PVS is not reclaiming space; version store can grow unboundedly, consuming tempdb or the PVS filegroup
- Fix: Identify long-running transactions blocking PVS cleanup:
SELECT * FROM sys.dm_tran_active_transactions WHERE transaction_begin_time < DATEADD(MINUTE,-10,GETUTCDATE()). Commit or roll back idle transactions. If the issue recurs, verify ADR is intentional — disable with ALTER DATABASE [db] SET ACCELERATED_DATABASE_RECOVERY = OFF if not needed.
E30 — IQP DOP Feedback Applied
- Trigger: Log contains
DOP feedback with adjusted or applied to — SQL 2022+ only; skip if compat level < 160
- Severity: Info — Intelligent Query Processing DOP Feedback has changed the degree of parallelism for a query. This is expected behavior but warrants review if performance degraded afterward.
- Fix: Query
sys.query_store_plan_feedback to see which queries received DOP adjustments and whether the feedback stabilized. If a DOP reduction caused regressions, force a plan or disable feedback for that query: EXEC sys.sp_query_store_set_hints @query_id = N'<id>', @query_hints = N'OPTION(USE HINT(''DISABLE_DOP_FEEDBACK''))'.
E31 — Ledger Verification Failure
- Trigger: Log contains
Ledger verification with failed or tamper detected — SQL 2022+ only
- Severity: Critical — Ledger table hash chain verification has detected a discrepancy; data integrity of the ledger table cannot be confirmed
- Fix: Run
sys.sp_verify_database_ledger and sys.sp_verify_database_ledger_from_digest_storage to determine scope. Preserve the ERRORLOG and all ledger digests for forensic analysis. Do not modify the affected tables until investigation is complete. Escalate to a security incident response process.
E32 — CE Feedback Model Version Change
- Trigger: Log contains
Cardinality Estimation feedback with model version change message — SQL 2022+ only; skip if compat level < 160
- Severity: Info — The Query Optimizer's CE feedback has promoted a learned model version for one or more queries. A sudden model-version change after a workload change can introduce plan regressions.
- Fix: Correlate the timestamp of the CE feedback message with any performance degradation in Query Store. Use
sys.query_store_plan_feedback to see affected queries. To disable CE feedback for a specific query: EXEC sys.sp_query_store_set_hints @query_id = N'<id>', @query_hints = N'OPTION(USE HINT(''DISABLE_CE_FEEDBACK''))'.
E33 — Azure Arc–Enabled SQL: Agent Disconnect
- Trigger: Log contains
Arc SQL extension with disconnected or heartbeat failure message — any SQL Server version with Azure Arc agent installed
- Severity: Warning — The Arc SQL extension agent has lost contact with the Azure control plane; Arc-based features (Microsoft Defender, automated backups, best practice assessments) are not functioning
- Fix: Check Arc agent health:
Get-Service -Name 'himds' and the SQL extension Get-Service -DisplayName 'Microsoft SQL Server Extension Service' (Windows; the service runs as NT SERVICE\SqlServerExtension — on Linux the service is named SqlServerExtension). There is no ArcSqlInstanceExtension service. Verify outbound connectivity to *.arc.azure.com and *.<region>.arcdataservices.com on port 443. Restart the extension service if it is stopped. Review Arc agent logs at %ProgramData%\GuestConfig\arc_policy_logs\ for detailed error messages.
Version-Aware Check Suppression
If the SQL Server version is known — from the startup banner in the ERRORLOG (e.g. SQL Server 2019 (RTM-CU18)...) or stated by the user — read VERSION_COMPATIBILITY.md (~/.claude/skills/VERSION_COMPATIBILITY.md if installed, or skills/VERSION_COMPATIBILITY.md from the repo). If unavailable, skip silently. For checks whose minimum version exceeds the instance version: verbose mode → log as SKIP (version: requires SQL 20XX+, instance is SQL 20YY); standard report → omit entirely. Do not suppress NOT ASSESSED rows from missing input — only suppress version-inapplicable checks.
Output Format
Structure the report as follows. Use this exact section order.
## SQL Server ERRORLOG Analysis
### Summary
- X Critical, Y Warnings, Z Info
- Time range: [first log entry datetime] – [last log entry datetime]
- SQL Server version: [version string from E28 startup line, or "Not found in provided excerpt"]
- Highest-risk finding: [check name and ID, e.g., "E2 — Lease Expiry"]
- Log coverage note: [single file / multiple files / partial excerpt — dates if known]
### Critical Issues
### [C1 — E2] Lease Expiry (2026-01-15 14:32:05)
- **Observed:** "lease between the availability group 'AG1' and the Windows Server Failover
Cluster has expired" at 14:32:05. Preceded by E15 (I/O slow) at 14:28:44 on
E:\Data\AG1_Primary.mdf.
- **Impact:** Unplanned AG failover triggered. AG1 primary role transferred to secondary.
Applications lost primary connection for the duration of the failover.
- **Fix:** Investigate I/O latency on E:\Data at 14:28 (see C2 — E15). Do not increase
LeaseTimeout without resolving the root cause I/O delay.
### Warnings
### [W1 — E1] AG Failover Event (2026-01-15 14:32:08)
...
### Info
### [I1 — E25] Trace Flag Active (startup)
...
### Passed Checks
| Check | Result |
|-------|--------|
| E9 — FAIL_PAGE_ALLOCATION | PASS — no FAIL_PAGE_ALLOCATION entries found |
| E16 — Database Corruption Warning | PASS — no checksum or torn-page errors found |
---
*Analyzed by: [state the AI model and version you are running as, e.g. "Claude Sonnet 4.6", "DeepSeek R1", "GPT-4o"] · [current date and time in the user's local timezone, or UTC if timezone is unknown, e.g. "2026-05-16 20:15 NZST"]*
Each finding label uses [C1], [W1], [I1] sequence numbering, with the check ID in
parentheses. Findings reference related checks by ID where one explains another
(e.g., "root cause of C1 — E2"). Passed Checks must list every check explicitly evaluated.
When a check cannot be evaluated (e.g., E18 with no backup log entries), state
"SKIP — no Database backed up entries in provided log window" rather than PASS or FAIL.
Notes
- ERRORLOG entries use local server time — note timezone if it differs from the analyst's context.
- Messages from
spid28s (or any spidNs) are system threads; Logon is the login auditing
thread; Backup is the backup thread.
- When multiple ERRORLOG files span a long window, the startup entry in each file signals the
beginning of a new SQL Server process (i.e., a restart occurred between files).
- The ERRORLOG does not record all events — OS-level events (WSFC partitions, disk controller
errors) appear only in the Windows Event Log and WSFC cluster log. Reference the companion
skill list below for those artifacts when ERRORLOG evidence points to external causes.
- Do not report a PASS for E18 if no
Database backed up entries are present — the absence
of backup log entries is itself an E18 signal for databases in FULL recovery. State clearly
which databases had backup evidence and which did not.
Section: Output Filters (--brief / --critical-only)
--brief — Omit the Passed Checks table and attribution footer. Output the Summary, Findings, and Prioritized Fix Sequence sections only. Use when a quick scan of what fired is all that's needed.
--critical-only — Suppress Warning and Info findings. Show only Critical findings. The Passed Checks table is also omitted. Use when triaging an incident and only actionable blockers matter.
Both flags can be combined: --brief --critical-only produces the Summary section plus Critical findings only.
When neither flag is present, produce the full report as documented above.
Section: Verbose Output (--verbose)
When the user's request includes --verbose, --trace, or the word verbose:
1. Append a ## Check Evaluation Log section after the Passed Checks table.
Include one row for every check in this skill's ruleset, in check-ID order:
| Check |
Evidence |
Threshold |
Result |
| [ID — Name] |
[key attribute(s) and value found, or "absent"] |
[threshold or condition] |
PASS / FIRE → [severity] / NOT ASSESSED |
Result conventions:
PASS — attribute present, threshold not met
**FIRE → Critical/Warning/Info** — threshold met; bold to distinguish from passes
NOT ASSESSED — required attribute absent from input
2. Save both files to the current working directory using the Write tool:
output//-/analysis.md ← full report
output//-/trace.md ← Check Evaluation Log
Derive <input-prefix>:
- Filename stem if a file path was provided (e.g.
horrible.sqlplan → horrible)
- First meaningful identifier from the artifact (top wait type, first table name, procedure name, etc.)
- Fallback:
run
Sanitize: alphanumeric + hyphens/underscores only, max 32 chars.
File headers:
analysis.md → # Analysis — <skill-name> / # Input: <first 80 chars> / # Generated: <UTC timestamp>
trace.md → # Check Evaluation Log — <skill-name> / # Input: <first 80 chars> / # Generated: <UTC timestamp>
Create directories as needed. When --verbose is not present, write nothing to disk.
Companion Skills
/sqlwait-review — correlate ERRORLOG memory and I/O signals (E9–E15) with
PAGEIOLATCH_SH, RESOURCE_SEMAPHORE, HADR_SYNC_COMMIT, and THREADPOOL wait dominance
/sqlplan-review + /sqlindex-advisor — analyze execution plans for queries that were
running during the incident window; high-cost queries during a memory or I/O event often
accelerate the failure
/sqlquerystore-review — identify plan regressions introduced after a post-incident restart
clears the plan cache, causing previously stable queries to recompile with bad plans
/tsql-review — review T-SQL source of stored procedures flagged during the incident as
high resource consumers before and after the failure
/sqldeadlock-review — if E22 (login failure burst) or connectivity errors coincide with error
1205 in application logs, analyze the deadlock XML from the system_health XE session
/sqlspn-review — when E22 (login failure burst) shows Kerberos-specific errors (17806, 17807,
error 0x8009030c) or "Login failed for user 'NT AUTHORITY\ANONYMOUS LOGON'", sqlspn-review
identifies the missing or duplicate SPN and delegation misconfiguration as root cause
mssql-performance-review — Orchestrator that routes mixed artifacts to multiple specialised skills (this one included), runs an adversarial root-cause check, and produces a single consolidated report with evidence chain, risk-rated fixes, and rollback. Use when you have several artifact types together or describe a symptom without knowing which skill to run.
1---2name: sqlerrorlog-review3description: Analyzes SQL Server ERRORLOG files for operational issues, availability group failures, memory pressure, I/O subsystem warnings, and security events. Use this skill whenever a SQL Server instance has experienced unexpected behavior, an AG failover, memory warnings, I/O latency alerts, or abnormal shutdown, and you need a structured timeline of what SQL Server recorded. Applies 33 checks (E1–E33) covering AG health, memory/resource pressure, I/O and storage, startup/shutdown, connectivity, configuration signals, and SQL 2019/2022 modern feature events.4---56# SQL Server ERRORLOG Review Skill78## Purpose910Parse and analyze SQL Server ERRORLOG content to surface operational warnings, high-availability11failures, resource pressure signals, security events, and configuration anomalies. Applies 3312checks (E1–E33) across six categories:1314- **E1–E8** — AG / High Availability: failovers, lease expiry, replica state changes, synchronization errors15- **E9–E14** — Memory and resource pressure: page allocation failures, OS paging, worker exhaustion, non-yielding schedulers16- **E15–E19** — I/O and storage: slow I/O subsystem, corruption warnings, tempdb exhaustion, log backup gaps, VLF proliferation17- **E20–E24** — Startup, shutdown, and connectivity: abnormal termination, restart cycling, login failure bursts, linked server errors18- **E25–E28** — Configuration and informational: trace flags, unconfigured max memory, log rotation gaps, version end-of-support19- **E29–E33** — SQL 2019/2022 modern features: ADR PVS cleanup stall, IQP DOP feedback, Ledger verification failure, CE feedback model change, Azure Arc agent disconnect2021## Input2223Accept any of:2425- **File path** — path to the SQL Server ERRORLOG file (default location:26 `C:\Program Files\Microsoft SQL Server\MSSQL<ver>.<inst>\MSSQL\Log\ERRORLOG`)27- **Inline paste** — raw ERRORLOG text pasted directly into chat; partial excerpts are valid28- **Natural language description** — describe the symptoms or paste selected log lines with context2930For best results, provide the current ERRORLOG and at least one prior log (`ERRORLOG.1`). When31only partial content is available, state which time range is covered.3233### Capture via T-SQL3435```sql36-- Read current ERRORLOG (0 = current, 1 = previous, 2 = the one before that)37EXEC xp_readerrorlog 0, 1; -- SQL Server log, current file38EXEC xp_readerrorlog 1, 1; -- SQL Server log, previous file3940-- Filter to AG-related messages only41EXEC xp_readerrorlog 0, 1, N'availability', NULL, NULL, NULL, N'desc';4243-- Filter to a time window (last 2 hours)44DECLARE @start DATETIME = DATEADD(HOUR, -2, GETDATE());45EXEC xp_readerrorlog 0, 1, NULL, NULL, @start, NULL, N'desc';46```4748### Column Reference4950| Column | Meaning |51|--------|---------|52| LogDate | Timestamp of the log entry (datetime2 precision) |53| ProcessInfo | SPID or system process (e.g., `spid28s`, `Logon`, `Backup`) |54| Text | Log message text |5556---5758## Thresholds Reference5960| Threshold | Value | Used by |61|-----------|-------|---------|62| Login failure burst — Warning | > 5 `Login failed` messages in any 5-min window | E22 |63| Login failure burst — Critical | > 20 `Login failed` messages in any 5-min window | E22 |64| Restart cycling | ≥ 2 SQL Server startup messages within 60 min | E21 |65| I/O slow built-in threshold | 15 seconds (SQL Server internal, non-configurable) | E15 |66| Log backup overdue — FULL/BULK_LOGGED | > 24 hr since last `Database backed up` entry | E18 |67| Log backup overdue — active log pressure signal | > 8 hr when `log_reuse_wait_desc = LOG_BACKUP` | E18 |6869---7071## AG / High Availability Checks (E1–E8)7273### E1 — AG Failover Event74- **Trigger:** Log contains `is changing roles from` or `is preparing to transition to` or `automatic failover`75 in the same entry or within the same minute as a role-change message; also `in response to a request from76 the Windows Server Failover Cluster`77- **Severity:** Warning — planned failover expected; Critical if the word `automatic` appears78 (unplanned loss of primary)79- **Fix:** For unplanned failovers, check E2 (lease expiry) and E6 (health check timeout) as80 probable root causes. For planned failovers in unexpected windows, review change-management81 records. Run `/sqlwait-review` on HADR_SYNC_COMMIT and HADR_WORK_QUEUE waits.8283### E2 — Lease Expiry84- **Trigger:** Log contains `lease between the availability group and the Windows Server Failover85 Cluster has expired` or `The lease of availability group` combined with `has expired`86- **Severity:** Critical — lease expiry is the most common root cause of unplanned AG failovers87- **Fix:** Investigate the time immediately before this entry for E15 (slow I/O), E1388 (non-yielding scheduler), or OS-level events. Common causes: storage latency spike causing89 the sp_server_diagnostics thread to miss its deadline, high CPU starvation, or WSFC network90 interruption. Increase `LeaseTimeout` in WSFC only as a temporary measure — fix the root cause.9192### E3 — Replica State Change93- **Trigger:** Log contains `The local replica of availability group ... is changing roles` or94 `is preparing to transition to the`95- **Severity:** Warning — state transitions are normal during planned operations; unexpected96 transitions during business hours warrant investigation97- **Fix:** Correlate the timestamp with E1 (failover), E2 (lease), or external WSFC events.98 If unplanned, check the Windows Event Log and WSFC cluster log for the triggering event.99100### E4 — AG Database Joining Failure101- **Trigger:** Log contains `Failed to join local availability database` or `The availability102 database ... is not in the correct state`103- **Severity:** Critical — the AG database is not receiving redo; secondary is running but not104 synchronized, providing false HA coverage105- **Fix:** Run `SELECT * FROM sys.dm_hadr_database_replica_states` to check106 `synchronization_state_desc` and `redo_queue_size`. If redo queue is growing, check disk I/O107 on the secondary. If the database is in `NOT SYNCHRONIZING`, re-join: `ALTER DATABASE [db]108 SET HADR AVAILABILITY GROUP = [ag_name]`.109110### E5 — Data Synchronisation Suspended111- **Trigger:** Log contains `Synchronization of this database ... has been suspended` or112 `Data movement for availability database ... has been suspended`113- **Severity:** Warning — a suspended database is not receiving log records; RPO clock is running114- **Fix:** Identify whether the suspension was manual (`ALTER DATABASE ... SET HADR SUSPEND`) or115 automatic (error-triggered). Check for E15/E16 (I/O or corruption) causing automatic suspension.116 Resume: `ALTER DATABASE [db] SET HADR RESUME`. Monitor redo queue.117118### E6 — AG Health Check Timeout119- **Trigger:** Log contains `availability group ... has failed to take necessary action within120 the time allotted` or `The availability group ... exceeded the health-check timeout`121- **Severity:** Critical — health-check failure directly precedes automatic failover; this entry122 combined with E1 confirms the full failover sequence123- **Fix:** Identify what the primary was doing at the time. E13 (non-yielding scheduler) or E9124 (page allocation failure) are common co-occurrences. The `HealthCheckTimeout` WSFC property125 controls sensitivity — do not increase it without fixing the underlying responsiveness problem.126127### E7 — Redo Thread Error128- **Trigger:** Log contains `An error occurred in the redo thread for database` or129 `Redo thread for database ... encountered error`130- **Severity:** Critical — the secondary redo thread has failed; the secondary is no longer131 applying log records and RPO is accumulating132- **Fix:** Note the error number in the log message. Common causes: corruption on the secondary133 (check E16), log record version mismatch after an upgrade, or disk full on secondary. For134 disk-full, free space and resume synchronization. For corruption, restore the secondary from135 a backup and re-seed.136137### E8 — Secondary Not Synchronising138- **Trigger:** Log contains `Waiting for redo catch-up` or mentions secondary redo queue in a139 warning context; or `log send queue` appearing repeatedly with growing values140- **Severity:** Warning — secondary is lagging; failover to this replica would result in data141 loss proportional to the redo queue depth142- **Fix:** Check network bandwidth between primary and secondary. Run143 `SELECT redo_queue_size, redo_rate FROM sys.dm_hadr_database_replica_states`. If redo rate <144 log generation rate, the secondary cannot keep up — review disk I/O on secondary (E15) or145 increase network bandwidth.146147---148149## Memory and Resource Pressure Checks (E9–E14)150151### E9 — FAIL_PAGE_ALLOCATION152- **Trigger:** Log contains `FAIL_PAGE_ALLOCATION` (exact string, case-insensitive)153- **Severity:** Critical — SQL Server could not satisfy an internal memory allocation; queries154 may have failed with out-of-memory errors; this entry often precedes OS paging (E10)155- **Fix:** Check `max server memory` configuration (E26). Run156 `SELECT type, pages_kb FROM sys.dm_os_memory_clerks ORDER BY pages_kb DESC` to identify157 which clerk is consuming the most memory. Consider reducing max server memory by 10–15% to158 leave headroom for OS and other processes.159160### E10 — OS Memory Pressure161- **Trigger:** Log contains `A significant part of sql server process memory has been paged out`162 or `Working set trim`163- **Severity:** Critical — Windows has paged SQL Server memory to disk under OS memory pressure;164 buffer pool pages are on disk, causing extreme I/O latency165- **Fix:** Reduce `max server memory` to allow OS headroom (leave at least 10% of RAM or 4 GB,166 whichever is greater). Enable `Lock Pages in Memory` (LPIM) to prevent paging for 64-bit SQL167 Server service account. Investigate other processes competing for RAM on the host.168169### E11 — Memory Insufficient (Resource Pool or Buffer Pool)170- **Trigger:** Log contains `There is insufficient memory available in the buffer pool` (error 802, buffer pool full) OR `There is insufficient system memory in resource pool` (error 701, Resource Governor pool exhausted) OR `Memory Manager: Memory node available memory is less than threshold`171- **Severity:** Critical — queries requiring memory grants are being denied; workload will stall on `RESOURCE_SEMAPHORE` waits172- **Fix:** Run `/sqlwait-review` and check for `RESOURCE_SEMAPHORE` dominance. Increase `max server memory` if physical RAM allows, or reduce `min memory per query` via Resource Governor. Identify large-grant queries with `/sqlplan-review` S2–S4. Error 802 (buffer pool) and error 701 (resource pool) require different fixes: 802 → increase max server memory or reduce buffer pool competition; 701 → adjust Resource Governor pool memory limits.173174### E12 — Worker Thread Exhaustion175- **Trigger:** Log contains `There are no more threads available to process new requests` or176 `Worker Thread ... has been waiting too long`177- **Severity:** Critical — new connections are being refused or queued; the instance is at178 maximum worker thread capacity179- **Fix:** Increase `max worker threads` via `sp_configure` only after identifying root cause.180 Common causes: blocking chains holding threads (check `sys.dm_exec_requests`), long-running181 queries, or undersized `max worker threads` for the workload. Run `/sqlwait-review` for182 `THREADPOOL` waits (V-checks).183184### E13 — Scheduler Non-Yielding185- **Trigger:** Log contains `Process appears to be non-yielding on Scheduler` or186 `A scheduler appears to be non-yielding`187- **Severity:** Critical — a thread is monopolising a scheduler without yielding; this blocks188 all other threads on that scheduler, degrades responsiveness, and can trigger AG health-check189 timeouts (E6) and lease expiry (E2)190- **Fix:** A memory dump is typically generated automatically. Look for a `.mdmp` file in the191 SQL Server Log directory matching the timestamp. Common causes: large in-memory sort, CLR192 call, XTP operation, or a bug in a specific build — check if a known hotfix applies for the193 version (E28). Consider enabling `DBCC TRACEON(8086)` on advice from Microsoft Support.194195### E14 — Memory Grant Timeout196- **Trigger:** Log contains `Memory grant request timed out` or197 `A request for memory failed with OOM (out of memory) status`198- **Severity:** Warning — a query could not acquire its requested memory grant within the199 timeout; it may have been killed or retried with a reduced grant, causing a spill to TempDb200- **Fix:** Capture the affected query and run `/sqlplan-review` for S2–S4 (memory grant checks).201 Update statistics to improve cardinality estimates. Use Resource Governor to cap grants for202 ad-hoc workloads. Check for E11 (resource pool exhaustion) as a co-trigger.203204---205206## I/O and Storage Checks (E15–E19)207208### E15 — I/O Subsystem Slow209- **Trigger:** Log contains `SQL Server has encountered` combined with `I/O requests taking210 longer than 15 seconds` (SQL Server's built-in slow I/O threshold)211- **Severity:** Critical — storage latency has exceeded the 15-second internal threshold;212 this is a primary trigger for AG lease expiry (E2) and health-check timeouts (E6)213- **Fix:** Note the file path and database in the message. Investigate storage subsystem: check214 disk queue length, RAID controller cache status, SAN/NVMe latency metrics, and any concurrent215 backup or maintenance operations competing for I/O. If on a VM, check storage IOPS limits.216 Run `/sqlwait-review` for `PAGEIOLATCH_SH` and `PAGEIOLATCH_EX` dominance.217218### E16 — Database Corruption Warning219- **Trigger:** Log contains `checksum mismatch`, `torn page`, `consistency errors detected`,220 `DBCC CHECKDB found` with error counts > 0, `Error: 823` (OS-level I/O failure — Windows API221 returned an error), `Error: 824` (logical consistency check failure — Windows I/O succeeded but222 SQL detected corruption on the page), or `Error: 825` (read succeeded after retry — transient223 storage issue; Warning severity; indicates potential hardware degradation)224- **Severity:** Critical for Msg 823 and Msg 824 (data corruption confirmed); Warning for Msg 825225 (read ultimately succeeded but is a precursor to harder failures — investigate storage hardware226 immediately); Critical if DBCC CHECKDB reports allocation or consistency errors227- **Fix:** Run `DBCC CHECKDB ([database]) WITH NO_INFOMSGS` immediately to assess scope. Do228 not attempt to repair until a current, verified backup exists. For `REPAIR_ALLOW_DATA_LOSS`,229 treat it as a last resort — restore from backup is always preferable. Investigate E15 (I/O230 latency) and storage hardware health as root causes.231232### E17 — TempDB Space Exhaustion233- **Trigger:** Log contains `Could not allocate space` combined with `in database 'tempdb'`234 or `tempdb is full` or `tempdb ran out of space`235- **Severity:** Critical — queries requiring temporary space (sorts, hashes, spools, row236 versioning) are failing; error 1105 is returned to applications237- **Fix:** Immediately: `DBCC SHRINKFILE` on tempdb data files to recover any unused allocated238 space, or add a tempdb data file. Long term: investigate which query is consuming tempdb239 (check `sys.dm_db_session_space_usage`). Run `/sqlplan-review` for N41–N43 (spill operators).240 Consider pre-allocating tempdb to expected working size at startup.241242### E18 — Log Backup Overdue243- **Trigger:** Gap between consecutive `Database backed up` entries for the same database244 exceeds the threshold for that recovery model. For databases in FULL or BULK_LOGGED recovery,245 flag if the gap exceeds 24 hours; flag more urgently if log backup entries are absent while246 other evidence suggests active transaction log growth247- **Severity:** Warning — log space will grow unboundedly without log backups; in a FULL248 recovery database, the log cannot be truncated until backed up249- **Fix:** Run a log backup immediately: `BACKUP LOG [database] TO DISK = N'path\logbackup.bak'`.250 Verify the SQL Agent log backup job is scheduled and enabled. Check `sys.databases` column251 `log_reuse_wait_desc` — if `LOG_BACKUP`, the log is waiting for a backup to allow truncation.252253### E19 — VLF Proliferation Signal254- **Trigger:** Log shows repeated `autogrow` events on transaction log files (multiple autogrow255 completions in the log window), or the database log has grown significantly between ERRORLOG256 entries — inferred from repeated log file path growth messages257- **Severity:** Info — excessive VLFs degrade recovery time and log-backup performance; auto-grow258 events indicate the log was not sized for the workload259- **Fix:** Shrink and pre-size the log: set the initial log file size to cover expected working260 set and disable autogrow on the log (or set a large, infrequent growth increment). Run261 `DBCC LOGINFO ([database])` to count current VLFs — if > 1,000, shrink and re-expand in one262 step. Align with E18 (log backup cadence) to ensure the log truncates regularly.263264---265266## Startup, Shutdown, and Connectivity Checks (E20–E24)267268### E20 — Abnormal Shutdown269- **Trigger:** Log contains `SQL Server is terminating` or `SQL Server has encountered` combined270 with `stack dump` or shutdown messages, without a preceding graceful shutdown marker271 (`SQL Server is terminating due to a system shutdown request` at the end of the prior log file)272- **Severity:** Critical — the instance crashed rather than shut down cleanly; uncommitted273 transactions were rolled back on restart; any in-flight work is lost274- **Fix:** Check the Windows Event Log (`Application` and `System` sources) for the crash275 timestamp. Look for a dump file in the SQL Server Log directory. If the crash occurred276 mid-transaction in an AG, check whether secondary databases advanced beyond the primary277 (split-brain risk). Engage Microsoft Support with the minidump if the crash is reproducible.278279### E21 — Repeated Restarts280- **Trigger:** ERRORLOG or combined ERRORLOG + ERRORLOG.1 contains ≥ 2 SQL Server startup281 messages (lines containing `SQL Server is starting` or `This instance of SQL Server last282 reported using a process ID`) within a 60-minute window283- **Severity:** Critical — the instance is crash-looping; each restart drops all plan cache and284 connection state; applications experience repeated connection failures285- **Fix:** Check E20 (abnormal shutdown) for the crash cause between restarts. If the instance286 is restarting due to a failed startup condition (e.g., tempdb creation failure, master database287 corruption, or xp_cmdshell misconfiguration), resolve the startup error first. Enable Windows288 `Automatic Recovery` only after identifying the underlying fault.289290### E22 — Login Failure Burst291- **Trigger:** Count of `Login failed` entries exceeds the threshold within a 5-minute rolling292 window — see Thresholds Reference for Warning and Critical levels293- **Severity:** Warning if > 5 failures in 5 min; Critical if > 20 failures in 5 min294- **Fix:** Identify the `ClientConnectionID` and source IP in the failure messages. A burst from295 one account likely indicates a misconfigured application connection string after a password296 rotation. A burst from many accounts may indicate a brute-force attempt. For brute-force:297 enable SQL Server Audit or Extended Events on `Failed Logins` and block the source IP at the298 network layer. Ensure `LOGINAUDIT` is set to `Failed logins only` or `Both` in Server299 properties so future bursts appear in the ERRORLOG.300301### E23 — Linked Server Error302- **Trigger:** Log contains `OLE DB provider` combined with `reported an error` or303 `Cannot obtain the required interface` for a linked server provider304- **Severity:** Warning — distributed queries or cross-server stored procedures using this305 linked server will fail until the provider error is resolved306- **Fix:** Identify the linked server name and provider from the error text. Common causes:307 target server unavailable, credential expiry, or OLE DB provider version mismatch. Test308 connectivity: `EXEC sp_testlinkedserver [linked_server_name]`. If the provider is outdated,309 update it on the SQL Server host.310311### E24 — Connectivity Error312- **Trigger:** Log contains `A connection was successfully established with the server, but313 then an error occurred during the login process` or `The connection has been lost` or314 `A network-related or instance-specific error` in the ERRORLOG (as opposed to the client)315- **Severity:** Warning — SQL Server is logging errors from its own outbound connections316 (linked servers, distributed queries, SSISDB, mail, replication) or from incoming connections317 that dropped after TCP establishment318- **Fix:** Correlate the timestamp with E22 (login failures), network infrastructure changes,319 or TLS/SSL certificate renewals. If `TLS handshake` appears in the message, verify that320 the certificate in use has not expired and that the client supports the negotiated protocol.321322---323324## Configuration and Informational Checks (E25–E28)325326### E25 — Trace Flag Active327- **Trigger:** Log contains `Trace flag` combined with `is set` or `was enabled at startup`328 in startup messages329- **Severity:** Info — trace flags change engine behavior; document intent and verify they330 are still appropriate for the current SQL Server version331- **Fix:** List all active trace flags: `DBCC TRACESTATUS(-1)`. Common production trace flags332 and their intent: 1117/1118 (tempdb allocation — superseded in 2016+), 3226 (suppress333 successful backup log entries), 4199 (QO hotfixes). Remove trace flags that are no longer334 needed or that apply to behaviour fixed in a later CU.335336### E26 — Max Server Memory Default337- **Trigger:** Log contains startup line showing `max server memory` = 2147483647 MB, or the338 instance has been running with the default (unlimited) memory configuration — inferred from339 startup messages or the absence of an explicit `max server memory` setting entry340- **Severity:** Info — unlimited memory allows SQL Server to consume all available RAM, starving341 the OS and any other services, which can trigger E10 (OS paging)342- **Fix:** Set `max server memory` to total RAM minus OS headroom: leave at least 10% of RAM or343 4 GB (whichever is larger) for the OS. For example, on a 64 GB server:344 `EXEC sp_configure 'max server memory (MB)', 57344; RECONFIGURE;`345346### E27 — ERRORLOG Rotation Gap347- **Trigger:** Only a single ERRORLOG file is provided, covering a window shorter than 24 hours,348 with no prior context from ERRORLOG.1 or earlier349- **Severity:** Info — events before the current file (including the original startup, prior350 AG failovers, or earlier memory events) are not visible; findings may be incomplete351- **Fix:** Retrieve prior ERRORLOG files: `EXEC xp_readerrorlog 1, 1` through352 `EXEC xp_readerrorlog 6, 1` (SQL Server retains up to 6 prior logs by default, configurable353 in SSMS → Server Properties → Database Settings → Number of error log files). State in the354 report: "Analysis covers [start] – [end] only; prior events not available."355356### E28 — SQL Server Version357- **Trigger:** Startup line containing `Microsoft SQL Server 20XX` version string — present358 in every ERRORLOG at instance start359- **Severity:** Info — extract and evaluate: (1) is this build on extended support, mainstream360 support, or past end-of-support? (2) is this the latest CU for this major version?361- **Fix:** Compare the build number in the log against the Microsoft SQL Server build list.362 If past end-of-support (e.g., SQL 2014 Extended Support ended 2024-07-09 — 2019 was only its363 *mainstream* support end; SQL 2016 Extended Support ends 2026-07-14), plan364 upgrade. If not on the latest CU, evaluate whether open bugs fixed in later CUs are relevant365 to the observed issues. Report the version string verbatim in the Output Summary.366367## SQL 2019/2022 Modern Feature Checks (E29–E33)368369### E29 — ADR PVS Cleanup Stall370- **Trigger:** Log contains `Persistent Version Store cleanup` with `stall` or `unable to advance` — SQL 2019+; skip if compat level < 150371- **Severity:** Warning — ADR (Accelerated Database Recovery) PVS is not reclaiming space; version store can grow unboundedly, consuming tempdb or the PVS filegroup372- **Fix:** Identify long-running transactions blocking PVS cleanup: `SELECT * FROM sys.dm_tran_active_transactions WHERE transaction_begin_time < DATEADD(MINUTE,-10,GETUTCDATE())`. Commit or roll back idle transactions. If the issue recurs, verify ADR is intentional — disable with `ALTER DATABASE [db] SET ACCELERATED_DATABASE_RECOVERY = OFF` if not needed.373374### E30 — IQP DOP Feedback Applied375- **Trigger:** Log contains `DOP feedback` with `adjusted` or `applied to` — SQL 2022+ only; skip if compat level < 160376- **Severity:** Info — Intelligent Query Processing DOP Feedback has changed the degree of parallelism for a query. This is expected behavior but warrants review if performance degraded afterward.377- **Fix:** Query `sys.query_store_plan_feedback` to see which queries received DOP adjustments and whether the feedback stabilized. If a DOP reduction caused regressions, force a plan or disable feedback for that query: `EXEC sys.sp_query_store_set_hints @query_id = N'<id>', @query_hints = N'OPTION(USE HINT(''DISABLE_DOP_FEEDBACK''))'`.378379### E31 — Ledger Verification Failure380- **Trigger:** Log contains `Ledger verification` with `failed` or `tamper detected` — SQL 2022+ only381- **Severity:** Critical — Ledger table hash chain verification has detected a discrepancy; data integrity of the ledger table cannot be confirmed382- **Fix:** Run `sys.sp_verify_database_ledger` and `sys.sp_verify_database_ledger_from_digest_storage` to determine scope. Preserve the ERRORLOG and all ledger digests for forensic analysis. Do not modify the affected tables until investigation is complete. Escalate to a security incident response process.383384### E32 — CE Feedback Model Version Change385- **Trigger:** Log contains `Cardinality Estimation feedback` with `model version` change message — SQL 2022+ only; skip if compat level < 160386- **Severity:** Info — The Query Optimizer's CE feedback has promoted a learned model version for one or more queries. A sudden model-version change after a workload change can introduce plan regressions.387- **Fix:** Correlate the timestamp of the CE feedback message with any performance degradation in Query Store. Use `sys.query_store_plan_feedback` to see affected queries. To disable CE feedback for a specific query: `EXEC sys.sp_query_store_set_hints @query_id = N'<id>', @query_hints = N'OPTION(USE HINT(''DISABLE_CE_FEEDBACK''))'`.388389### E33 — Azure Arc–Enabled SQL: Agent Disconnect390- **Trigger:** Log contains `Arc SQL extension` with `disconnected` or `heartbeat` failure message — any SQL Server version with Azure Arc agent installed391- **Severity:** Warning — The Arc SQL extension agent has lost contact with the Azure control plane; Arc-based features (Microsoft Defender, automated backups, best practice assessments) are not functioning392- **Fix:** Check Arc agent health: `Get-Service -Name 'himds'` and the SQL extension `Get-Service -DisplayName 'Microsoft SQL Server Extension Service'` (Windows; the service runs as `NT SERVICE\SqlServerExtension` — on Linux the service is named `SqlServerExtension`). There is no `ArcSqlInstanceExtension` service. Verify outbound connectivity to `*.arc.azure.com` and `*.<region>.arcdataservices.com` on port 443. Restart the extension service if it is stopped. Review Arc agent logs at `%ProgramData%\GuestConfig\arc_policy_logs\` for detailed error messages.393394---395396## Version-Aware Check Suppression397398If the SQL Server version is known — from the startup banner in the ERRORLOG (e.g. `SQL Server 2019 (RTM-CU18)...`) or stated by the user — read `VERSION_COMPATIBILITY.md` (`~/.claude/skills/VERSION_COMPATIBILITY.md` if installed, or `skills/VERSION_COMPATIBILITY.md` from the repo). If unavailable, skip silently. For checks whose minimum version exceeds the instance version: verbose mode → log as `SKIP (version: requires SQL 20XX+, instance is SQL 20YY)`; standard report → omit entirely. Do not suppress `NOT ASSESSED` rows from missing input — only suppress version-inapplicable checks.399400---401402## Output Format403404Structure the report as follows. Use this exact section order.405406```407## SQL Server ERRORLOG Analysis408409### Summary410- X Critical, Y Warnings, Z Info411- Time range: [first log entry datetime] – [last log entry datetime]412- SQL Server version: [version string from E28 startup line, or "Not found in provided excerpt"]413- Highest-risk finding: [check name and ID, e.g., "E2 — Lease Expiry"]414- Log coverage note: [single file / multiple files / partial excerpt — dates if known]415416### Critical Issues417418### [C1 — E2] Lease Expiry (2026-01-15 14:32:05)419- **Observed:** "lease between the availability group 'AG1' and the Windows Server Failover420 Cluster has expired" at 14:32:05. Preceded by E15 (I/O slow) at 14:28:44 on421 E:\Data\AG1_Primary.mdf.422- **Impact:** Unplanned AG failover triggered. AG1 primary role transferred to secondary.423 Applications lost primary connection for the duration of the failover.424- **Fix:** Investigate I/O latency on E:\Data at 14:28 (see C2 — E15). Do not increase425 LeaseTimeout without resolving the root cause I/O delay.426427### Warnings428429### [W1 — E1] AG Failover Event (2026-01-15 14:32:08)430...431432### Info433434### [I1 — E25] Trace Flag Active (startup)435...436437### Passed Checks438439| Check | Result |440|-------|--------|441| E9 — FAIL_PAGE_ALLOCATION | PASS — no FAIL_PAGE_ALLOCATION entries found |442| E16 — Database Corruption Warning | PASS — no checksum or torn-page errors found |443444---445*Analyzed by: [state the AI model and version you are running as, e.g. "Claude Sonnet 4.6", "DeepSeek R1", "GPT-4o"] · [current date and time in the user's local timezone, or UTC if timezone is unknown, e.g. "2026-05-16 20:15 NZST"]*446```447448Each finding label uses `[C1]`, `[W1]`, `[I1]` sequence numbering, with the check ID in449parentheses. Findings reference related checks by ID where one explains another450(e.g., "root cause of C1 — E2"). Passed Checks must list every check explicitly evaluated.451When a check cannot be evaluated (e.g., E18 with no backup log entries), state452"SKIP — no `Database backed up` entries in provided log window" rather than PASS or FAIL.453454---455456## Notes457458- ERRORLOG entries use local server time — note timezone if it differs from the analyst's context.459- Messages from `spid28s` (or any `spidNs`) are system threads; `Logon` is the login auditing460 thread; `Backup` is the backup thread.461- When multiple ERRORLOG files span a long window, the startup entry in each file signals the462 beginning of a new SQL Server process (i.e., a restart occurred between files).463- The ERRORLOG does not record all events — OS-level events (WSFC partitions, disk controller464 errors) appear only in the Windows Event Log and WSFC cluster log. Reference the companion465 skill list below for those artifacts when ERRORLOG evidence points to external causes.466- Do not report a PASS for E18 if no `Database backed up` entries are present — the absence467 of backup log entries is itself an E18 signal for databases in FULL recovery. State clearly468 which databases had backup evidence and which did not.469470---471472### Section: Output Filters (--brief / --critical-only)473474**`--brief`** — Omit the Passed Checks table and attribution footer. Output the Summary, Findings, and Prioritized Fix Sequence sections only. Use when a quick scan of what fired is all that's needed.475476**`--critical-only`** — Suppress Warning and Info findings. Show only Critical findings. The Passed Checks table is also omitted. Use when triaging an incident and only actionable blockers matter.477478Both flags can be combined: `--brief --critical-only` produces the Summary section plus Critical findings only.479480When neither flag is present, produce the full report as documented above.481482---483484### Section: Verbose Output (--verbose)485486When the user's request includes `--verbose`, `--trace`, or the word `verbose`:487488**1. Append a `## Check Evaluation Log` section** after the Passed Checks table.489490Include one row for every check in this skill's ruleset, in check-ID order:491492| Check | Evidence | Threshold | Result |493|-------|----------|-----------|--------|494| [ID — Name] | [key attribute(s) and value found, or "absent"] | [threshold or condition] | PASS / **FIRE → [severity]** / NOT ASSESSED |495496Result conventions:497- `PASS` — attribute present, threshold not met498- `**FIRE → Critical/Warning/Info**` — threshold met; bold to distinguish from passes499- `NOT ASSESSED` — required attribute absent from input500501**2. Save both files** to the current working directory using the Write tool:502503 output/<skill-name>/<YYYY-MM-DD-HHmmss>-<input-prefix>/analysis.md ← full report504 output/<skill-name>/<YYYY-MM-DD-HHmmss>-<input-prefix>/trace.md ← Check Evaluation Log505506Derive `<input-prefix>`:5071. Filename stem if a file path was provided (e.g. `horrible.sqlplan` → `horrible`)5082. First meaningful identifier from the artifact (top wait type, first table name, procedure name, etc.)5093. Fallback: `run`510Sanitize: alphanumeric + hyphens/underscores only, max 32 chars.511512File headers:513 analysis.md → `# Analysis — <skill-name> / # Input: <first 80 chars> / # Generated: <UTC timestamp>`514 trace.md → `# Check Evaluation Log — <skill-name> / # Input: <first 80 chars> / # Generated: <UTC timestamp>`515516Create directories as needed. When `--verbose` is not present, write nothing to disk.517518---519520## Companion Skills521522- `/sqlwait-review` — correlate ERRORLOG memory and I/O signals (E9–E15) with523 `PAGEIOLATCH_SH`, `RESOURCE_SEMAPHORE`, `HADR_SYNC_COMMIT`, and `THREADPOOL` wait dominance524- `/sqlplan-review` + `/sqlindex-advisor` — analyze execution plans for queries that were525 running during the incident window; high-cost queries during a memory or I/O event often526 accelerate the failure527- `/sqlquerystore-review` — identify plan regressions introduced after a post-incident restart528 clears the plan cache, causing previously stable queries to recompile with bad plans529- `/tsql-review` — review T-SQL source of stored procedures flagged during the incident as530 high resource consumers before and after the failure531- `/sqldeadlock-review` — if E22 (login failure burst) or connectivity errors coincide with error532 1205 in application logs, analyze the deadlock XML from the `system_health` XE session533- `/sqlspn-review` — when E22 (login failure burst) shows Kerberos-specific errors (17806, 17807,534 error 0x8009030c) or "Login failed for user 'NT AUTHORITY\ANONYMOUS LOGON'", sqlspn-review535 identifies the missing or duplicate SPN and delegation misconfiguration as root cause536537- **mssql-performance-review** — Orchestrator that routes mixed artifacts to multiple specialised skills (this one included), runs an adversarial root-cause check, and produces a single consolidated report with evidence chain, risk-rated fixes, and rollback. Use when you have several artifact types together or describe a symptom without knowing which skill to run.