SQL Server ERRORLOG Review Skill
Purpose
Parse and analyze SQL Server ERRORLOG content to surface operational warnings, high-availability
failures, resource pressure signals, security events, and configuration anomalies. Applies 28
checks (E1–E28) across five categories:
- E1–E8 — AG / High Availability: failovers, lease expiry, replica state changes, synchronization errors
- E9–E14 — Memory and resource pressure: page allocation failures, OS paging, worker exhaustion, non-yielding schedulers
- E15–E19 — I/O and storage: slow I/O subsystem, corruption warnings, tempdb exhaustion, log backup gaps, VLF proliferation
- E20–E24 — Startup, shutdown, and connectivity: abnormal termination, restart cycling, login failure bursts, linked server errors
- E25–E28 — Configuration and informational: trace flags, unconfigured max memory, log rotation gaps, version end-of-support
Input
Accept any of:
- File path — path to the SQL Server ERRORLOG file (default location:
C:\Program Files\Microsoft SQL Server\MSSQL<ver>.<inst>\MSSQL\Log\ERRORLOG)
- Inline paste — raw ERRORLOG text pasted directly into chat; partial excerpts are valid
- Natural language description — describe the symptoms or paste selected log lines with context
For best results, provide the current ERRORLOG and at least one prior log (ERRORLOG.1). When
only partial content is available, state which time range is covered.
Capture via T-SQL
-- Read current ERRORLOG (0 = current, 1 = previous, 2 = the one before that)
EXEC xp_readerrorlog 0, 1; -- SQL Server log, current file
EXEC xp_readerrorlog 1, 1; -- SQL Server log, previous file
-- Filter to AG-related messages only
EXEC xp_readerrorlog 0, 1, N'availability', NULL, NULL, NULL, N'desc';
-- Filter to a time window (last 2 hours)
DECLARE @start DATETIME = DATEADD(HOUR, -2, GETDATE());
EXEC xp_readerrorlog 0, 1, NULL, NULL, @start, NULL, N'desc';
Column Reference
| Column |
Meaning |
| LogDate |
Timestamp of the log entry (datetime2 precision) |
| ProcessInfo |
SPID or system process (e.g., spid28s, Logon, Backup) |
| Text |
Log message text |
Thresholds Reference
| Threshold |
Value |
Used by |
| Login failure burst — Warning |
> 5 Login failed messages in any 5-min window |
E22 |
| Login failure burst — Critical |
> 20 Login failed messages in any 5-min window |
E22 |
| Restart cycling |
≥ 2 SQL Server startup messages within 60 min |
E21 |
| I/O slow built-in threshold |
15 seconds (SQL Server internal, non-configurable) |
E15 |
| Log backup overdue — FULL/BULK_LOGGED |
> 24 hr since last Database backed up entry |
E18 |
| Log backup overdue — active log pressure signal |
> 8 hr when log_reuse_wait_desc = LOG_BACKUP |
E18 |
AG / High Availability Checks (E1–E8)
E1 — AG Failover Event
- Trigger: Log contains
performing a planned role change or automatic failover in the same
entry or within the same minute as a role-change message; also in response to a request from the Windows Server Failover Cluster
- Severity: Warning — planned failover expected; Critical if the word
automatic appears
(unplanned loss of primary)
- Fix: For unplanned failovers, check E2 (lease expiry) and E6 (health check timeout) as
probable root causes. For planned failovers in unexpected windows, review change-management
records. Run
/sqlwait-review on HADR_SYNC_COMMIT and HADR_WORK_QUEUE waits.
E2 — Lease Expiry
- Trigger: Log contains
lease between the availability group and the Windows Server Failover Cluster has expired or The lease of availability group combined with has expired
- Severity: Critical — lease expiry is the most common root cause of unplanned AG failovers
- Fix: Investigate the time immediately before this entry for E15 (slow I/O), E13
(non-yielding scheduler), or OS-level events. Common causes: storage latency spike causing
the sp_server_diagnostics thread to miss its deadline, high CPU starvation, or WSFC network
interruption. Increase
LeaseTimeout in WSFC only as a temporary measure — fix the root cause.
E3 — Replica State Change
- Trigger: Log contains
The local replica of availability group ... is changing roles or
is preparing to transition to the
- Severity: Warning — state transitions are normal during planned operations; unexpected
transitions during business hours warrant investigation
- Fix: Correlate the timestamp with E1 (failover), E2 (lease), or external WSFC events.
If unplanned, check the Windows Event Log and WSFC cluster log for the triggering event.
E4 — AG Database Joining Failure
- Trigger: Log contains
Failed to join local availability database or The availability database ... is not in the correct state
- Severity: Critical — the AG database is not receiving redo; secondary is running but not
synchronized, providing false HA coverage
- Fix: Run
SELECT * FROM sys.dm_hadr_database_replica_states to check
synchronization_state_desc and redo_queue_size. If redo queue is growing, check disk I/O
on the secondary. If the database is in NOT SYNCHRONIZING, re-join: ALTER DATABASE [db] SET HADR AVAILABILITY GROUP = [ag_name].
E5 — Data Synchronisation Suspended
- Trigger: Log contains
Synchronization of this database ... has been suspended or
Data movement for availability database ... has been suspended
- Severity: Warning — a suspended database is not receiving log records; RPO clock is running
- Fix: Identify whether the suspension was manual (
ALTER DATABASE ... SET HADR SUSPEND) or
automatic (error-triggered). Check for E15/E16 (I/O or corruption) causing automatic suspension.
Resume: ALTER DATABASE [db] SET HADR RESUME. Monitor redo queue.
E6 — AG Health Check Timeout
- Trigger: Log contains
availability group ... has failed to take necessary action within the time allotted or The availability group ... exceeded the health-check timeout
- Severity: Critical — health-check failure directly precedes automatic failover; this entry
combined with E1 confirms the full failover sequence
- Fix: Identify what the primary was doing at the time. E13 (non-yielding scheduler) or E9
(page allocation failure) are common co-occurrences. The
HealthCheckTimeout WSFC property
controls sensitivity — do not increase it without fixing the underlying responsiveness problem.
E7 — Redo Thread Error
- Trigger: Log contains
An error occurred in the redo thread for database or
Redo thread for database ... encountered error
- Severity: Critical — the secondary redo thread has failed; the secondary is no longer
applying log records and RPO is accumulating
- Fix: Note the error number in the log message. Common causes: corruption on the secondary
(check E16), log record version mismatch after an upgrade, or disk full on secondary. For
disk-full, free space and resume synchronization. For corruption, restore the secondary from
a backup and re-seed.
E8 — Secondary Not Synchronising
- Trigger: Log contains
Waiting for redo catch-up or mentions secondary redo queue in a
warning context; or log send queue appearing repeatedly with growing values
- Severity: Warning — secondary is lagging; failover to this replica would result in data
loss proportional to the redo queue depth
- Fix: Check network bandwidth between primary and secondary. Run
SELECT redo_queue_size, redo_rate FROM sys.dm_hadr_database_replica_states. If redo rate <
log generation rate, the secondary cannot keep up — review disk I/O on secondary (E15) or
increase network bandwidth.
Memory and Resource Pressure Checks (E9–E14)
E9 — FAIL_PAGE_ALLOCATION
- Trigger: Log contains
FAIL_PAGE_ALLOCATION (exact string, case-insensitive)
- Severity: Critical — SQL Server could not satisfy an internal memory allocation; queries
may have failed with out-of-memory errors; this entry often precedes OS paging (E10)
- Fix: Check
max server memory configuration (E26). Run
SELECT type, pages_kb FROM sys.dm_os_memory_clerks ORDER BY pages_kb DESC to identify
which clerk is consuming the most memory. Consider reducing max server memory by 10–15% to
leave headroom for OS and other processes.
E10 — OS Memory Pressure
- Trigger: Log contains
A significant part of sql server process memory has been paged out
or Working set trim
- Severity: Critical — Windows has paged SQL Server memory to disk under OS memory pressure;
buffer pool pages are on disk, causing extreme I/O latency
- Fix: Reduce
max server memory to allow OS headroom (leave at least 10% of RAM or 4 GB,
whichever is greater). Enable Lock Pages in Memory (LPIM) to prevent paging for 64-bit SQL
Server service account. Investigate other processes competing for RAM on the host.
E11 — Buffer Pool Insufficient
- Trigger: Log contains
There is insufficient system memory in resource pool or
Memory Manager: Memory node available memory is less than threshold
- Severity: Critical — queries requiring memory grants are being denied; workload will
stall on
RESOURCE_SEMAPHORE waits
- Fix: Run
/sqlwait-review and check for RESOURCE_SEMAPHORE dominance. Increase
max server memory if physical RAM allows, or reduce min memory per query via Resource
Governor. Identify large-grant queries with /sqlplan-review S2–S4.
E12 — Worker Thread Exhaustion
- Trigger: Log contains
There are no more threads available to process new requests or
Worker Thread ... has been waiting too long
- Severity: Critical — new connections are being refused or queued; the instance is at
maximum worker thread capacity
- Fix: Increase
max worker threads via sp_configure only after identifying root cause.
Common causes: blocking chains holding threads (check sys.dm_exec_requests), long-running
queries, or undersized max worker threads for the workload. Run /sqlwait-review for
THREADPOOL waits (V-checks).
E13 — Scheduler Non-Yielding
- Trigger: Log contains
Process appears to be non-yielding on Scheduler or
A scheduler appears to be non-yielding
- Severity: Critical — a thread is monopolising a scheduler without yielding; this blocks
all other threads on that scheduler, degrades responsiveness, and can trigger AG health-check
timeouts (E6) and lease expiry (E2)
- Fix: A memory dump is typically generated automatically. Look for a
.mdmp file in the
SQL Server Log directory matching the timestamp. Common causes: large in-memory sort, CLR
call, XTP operation, or a bug in a specific build — check if a known hotfix applies for the
version (E28). Consider enabling DBCC TRACEON(8086) on advice from Microsoft Support.
E14 — Memory Grant Timeout
- Trigger: Log contains
Memory grant request timed out or
A request for memory failed with OOM (out of memory) status
- Severity: Warning — a query could not acquire its requested memory grant within the
timeout; it may have been killed or retried with a reduced grant, causing a spill to TempDb
- Fix: Capture the affected query and run
/sqlplan-review for S2–S4 (memory grant checks).
Update statistics to improve cardinality estimates. Use Resource Governor to cap grants for
ad-hoc workloads. Check for E11 (resource pool exhaustion) as a co-trigger.
I/O and Storage Checks (E15–E19)
E15 — I/O Subsystem Slow
- Trigger: Log contains
SQL Server has encountered combined with I/O requests taking longer than 15 seconds (SQL Server's built-in slow I/O threshold)
- Severity: Critical — storage latency has exceeded the 15-second internal threshold;
this is a primary trigger for AG lease expiry (E2) and health-check timeouts (E6)
- Fix: Note the file path and database in the message. Investigate storage subsystem: check
disk queue length, RAID controller cache status, SAN/NVMe latency metrics, and any concurrent
backup or maintenance operations competing for I/O. If on a VM, check storage IOPS limits.
Run
/sqlwait-review for PAGEIOLATCH_SH and PAGEIOLATCH_EX dominance.
E16 — Database Corruption Warning
- Trigger: Log contains
checksum mismatch, torn page, consistency errors detected,
or DBCC CHECKDB found with error counts > 0
- Severity: Critical — data corruption has been detected; backup integrity is unknown until
verified; the affected database may be inaccessible or returning wrong results
- Fix: Run
DBCC CHECKDB ([database]) WITH NO_INFOMSGS immediately to assess scope. Do
not attempt to repair until a current, verified backup exists. For REPAIR_ALLOW_DATA_LOSS,
treat it as a last resort — restore from backup is always preferable. Investigate E15 (I/O
latency) and storage hardware health as root causes.
E17 — TempDB Space Exhaustion
- Trigger: Log contains
Could not allocate space combined with in database 'tempdb'
or tempdb is full or tempdb ran out of space
- Severity: Critical — queries requiring temporary space (sorts, hashes, spools, row
versioning) are failing; error 1105 is returned to applications
- Fix: Immediately:
DBCC SHRINKFILE on tempdb data files to recover any unused allocated
space, or add a tempdb data file. Long term: investigate which query is consuming tempdb
(check sys.dm_db_session_space_usage). Run /sqlplan-review for N41–N43 (spill operators).
Consider pre-allocating tempdb to expected working size at startup.
E18 — Log Backup Overdue
- Trigger: Gap between consecutive
Database backed up entries for the same database
exceeds the threshold for that recovery model. For databases in FULL or BULK_LOGGED recovery,
flag if the gap exceeds 24 hours; flag more urgently if log backup entries are absent while
other evidence suggests active transaction log growth
- Severity: Warning — log space will grow unboundedly without log backups; in a FULL
recovery database, the log cannot be truncated until backed up
- Fix: Run a log backup immediately:
BACKUP LOG [database] TO DISK = N'path\logbackup.bak'.
Verify the SQL Agent log backup job is scheduled and enabled. Check sys.databases column
log_reuse_wait_desc — if LOG_BACKUP, the log is waiting for a backup to allow truncation.
E19 — VLF Proliferation Signal
- Trigger: Log shows repeated
autogrow events on transaction log files (multiple autogrow
completions in the log window), or the database log has grown significantly between ERRORLOG
entries — inferred from repeated log file path growth messages
- Severity: Info — excessive VLFs degrade recovery time and log-backup performance; auto-grow
events indicate the log was not sized for the workload
- Fix: Shrink and pre-size the log: set the initial log file size to cover expected working
set and disable autogrow on the log (or set a large, infrequent growth increment). Run
DBCC LOGINFO ([database]) to count current VLFs — if > 1,000, shrink and re-expand in one
step. Align with E18 (log backup cadence) to ensure the log truncates regularly.
Startup, Shutdown, and Connectivity Checks (E20–E24)
E20 — Abnormal Shutdown
- Trigger: Log contains
SQL Server is terminating or SQL Server has encountered combined
with stack dump or shutdown messages, without a preceding graceful shutdown marker
(SQL Server is terminating due to a system shutdown request at the end of the prior log file)
- Severity: Critical — the instance crashed rather than shut down cleanly; uncommitted
transactions were rolled back on restart; any in-flight work is lost
- Fix: Check the Windows Event Log (
Application and System sources) for the crash
timestamp. Look for a dump file in the SQL Server Log directory. If the crash occurred
mid-transaction in an AG, check whether secondary databases advanced beyond the primary
(split-brain risk). Engage Microsoft Support with the minidump if the crash is reproducible.
E21 — Repeated Restarts
- Trigger: ERRORLOG or combined ERRORLOG + ERRORLOG.1 contains ≥ 2 SQL Server startup
messages (lines containing
SQL Server is starting or This instance of SQL Server last reported using a process ID) within a 60-minute window
- Severity: Critical — the instance is crash-looping; each restart drops all plan cache and
connection state; applications experience repeated connection failures
- Fix: Check E20 (abnormal shutdown) for the crash cause between restarts. If the instance
is restarting due to a failed startup condition (e.g., tempdb creation failure, master database
corruption, or xp_cmdshell misconfiguration), resolve the startup error first. Enable Windows
Automatic Recovery only after identifying the underlying fault.
E22 — Login Failure Burst
- Trigger: Count of
Login failed entries exceeds the threshold within a 5-minute rolling
window — see Thresholds Reference for Warning and Critical levels
- Severity: Warning if > 5 failures in 5 min; Critical if > 20 failures in 5 min
- Fix: Identify the
ClientConnectionID and source IP in the failure messages. A burst from
one account likely indicates a misconfigured application connection string after a password
rotation. A burst from many accounts may indicate a brute-force attempt. For brute-force:
enable SQL Server Audit or Extended Events on Failed Logins and block the source IP at the
network layer. Ensure LOGINAUDIT is set to Failed logins only or Both in Server
properties so future bursts appear in the ERRORLOG.
E23 — Linked Server Error
- Trigger: Log contains
OLE DB provider combined with reported an error or
Cannot obtain the required interface for a linked server provider
- Severity: Warning — distributed queries or cross-server stored procedures using this
linked server will fail until the provider error is resolved
- Fix: Identify the linked server name and provider from the error text. Common causes:
target server unavailable, credential expiry, or OLE DB provider version mismatch. Test
connectivity:
EXEC sp_testlinkedserver [linked_server_name]. If the provider is outdated,
update it on the SQL Server host.
E24 — Connectivity Error
- Trigger: Log contains
A connection was successfully established with the server, but then an error occurred during the login process or The connection has been lost or
A network-related or instance-specific error in the ERRORLOG (as opposed to the client)
- Severity: Warning — SQL Server is logging errors from its own outbound connections
(linked servers, distributed queries, SSISDB, mail, replication) or from incoming connections
that dropped after TCP establishment
- Fix: Correlate the timestamp with E22 (login failures), network infrastructure changes,
or TLS/SSL certificate renewals. If
TLS handshake appears in the message, verify that
the certificate in use has not expired and that the client supports the negotiated protocol.
Configuration and Informational Checks (E25–E28)
E25 — Trace Flag Active
- Trigger: Log contains
Trace flag combined with is set or was enabled at startup
in startup messages
- Severity: Info — trace flags change engine behavior; document intent and verify they
are still appropriate for the current SQL Server version
- Fix: List all active trace flags:
DBCC TRACESTATUS(-1). Common production trace flags
and their intent: 1117/1118 (tempdb allocation — superseded in 2016+), 3226 (suppress
successful backup log entries), 4199 (QO hotfixes). Remove trace flags that are no longer
needed or that apply to behaviour fixed in a later CU.
E26 — Max Server Memory Default
- Trigger: Log contains startup line showing
max server memory = 2147483647 MB, or the
instance has been running with the default (unlimited) memory configuration — inferred from
startup messages or the absence of an explicit max server memory setting entry
- Severity: Info — unlimited memory allows SQL Server to consume all available RAM, starving
the OS and any other services, which can trigger E10 (OS paging)
- Fix: Set
max server memory to total RAM minus OS headroom: leave at least 10% of RAM or
4 GB (whichever is larger) for the OS. For example, on a 64 GB server:
EXEC sp_configure 'max server memory (MB)', 57344; RECONFIGURE;
E27 — ERRORLOG Rotation Gap
- Trigger: Only a single ERRORLOG file is provided, covering a window shorter than 24 hours,
with no prior context from ERRORLOG.1 or earlier
- Severity: Info — events before the current file (including the original startup, prior
AG failovers, or earlier memory events) are not visible; findings may be incomplete
- Fix: Retrieve prior ERRORLOG files:
EXEC xp_readerrorlog 1, 1 through
EXEC xp_readerrorlog 6, 1 (SQL Server retains up to 6 prior logs by default, configurable
in SSMS → Server Properties → Database Settings → Number of error log files). State in the
report: "Analysis covers [start] – [end] only; prior events not available."
E28 — SQL Server Version
- Trigger: Startup line containing
Microsoft SQL Server 20XX version string — present
in every ERRORLOG at instance start
- Severity: Info — extract and evaluate: (1) is this build on extended support, mainstream
support, or past end-of-support? (2) is this the latest CU for this major version?
- Fix: Compare the build number in the log against the Microsoft SQL Server build list.
If past end-of-support (e.g., SQL 2014 EOL 2019-07-09, SQL 2016 EOL 2026-07-14), plan
upgrade. If not on the latest CU, evaluate whether open bugs fixed in later CUs are relevant
to the observed issues. Report the version string verbatim in the Output Summary.
Output Format
Structure the report as follows. Use this exact section order.
## SQL Server ERRORLOG Analysis
### Summary
- X Critical, Y Warnings, Z Info
- Time range: [first log entry datetime] – [last log entry datetime]
- SQL Server version: [version string from E28 startup line, or "Not found in provided excerpt"]
- Highest-risk finding: [check name and ID, e.g., "E2 — Lease Expiry"]
- Log coverage note: [single file / multiple files / partial excerpt — dates if known]
### Critical Issues
### [C1 — E2] Lease Expiry (2026-01-15 14:32:05)
- **Observed:** "lease between the availability group 'AG1' and the Windows Server Failover
Cluster has expired" at 14:32:05. Preceded by E15 (I/O slow) at 14:28:44 on
E:\Data\AG1_Primary.mdf.
- **Impact:** Unplanned AG failover triggered. AG1 primary role transferred to secondary.
Applications lost primary connection for the duration of the failover.
- **Fix:** Investigate I/O latency on E:\Data at 14:28 (see C2 — E15). Do not increase
LeaseTimeout without resolving the root cause I/O delay.
### Warnings
### [W1 — E1] AG Failover Event (2026-01-15 14:32:08)
...
### Info
### [I1 — E25] Trace Flag Active (startup)
...
### Passed Checks
| Check | Result |
|-------|--------|
| E9 — FAIL_PAGE_ALLOCATION | PASS — no FAIL_PAGE_ALLOCATION entries found |
| E16 — Database Corruption Warning | PASS — no checksum or torn-page errors found |
Each finding label uses [C1], [W1], [I1] sequence numbering, with the check ID in
parentheses. Findings reference related checks by ID where one explains another
(e.g., "root cause of C1 — E2"). Passed Checks must list every check explicitly evaluated.
When a check cannot be evaluated (e.g., E18 with no backup log entries), state
"SKIP — no Database backed up entries in provided log window" rather than PASS or FAIL.
Notes
- ERRORLOG entries use local server time — note timezone if it differs from the analyst's context.
- Messages from
spid28s (or any spidNs) are system threads; Logon is the login auditing
thread; Backup is the backup thread.
- When multiple ERRORLOG files span a long window, the startup entry in each file signals the
beginning of a new SQL Server process (i.e., a restart occurred between files).
- The ERRORLOG does not record all events — OS-level events (WSFC partitions, disk controller
errors) appear only in the Windows Event Log and WSFC cluster log. Reference the companion
skill list below for those artifacts when ERRORLOG evidence points to external causes.
- Do not report a PASS for E18 if no
Database backed up entries are present — the absence
of backup log entries is itself an E18 signal for databases in FULL recovery. State clearly
which databases had backup evidence and which did not.
Companion Skills
/sqlwait-review — correlate ERRORLOG memory and I/O signals (E9–E15) with
PAGEIOLATCH_SH, RESOURCE_SEMAPHORE, HADR_SYNC_COMMIT, and THREADPOOL wait dominance
/sqlplan-review + /sqlplan-index-advisor — analyze execution plans for queries that were
running during the incident window; high-cost queries during a memory or I/O event often
accelerate the failure
/query-store-review — identify plan regressions introduced after a post-incident restart
clears the plan cache, causing previously stable queries to recompile with bad plans
/tsql-review — review T-SQL source of stored procedures flagged during the incident as
high resource consumers before and after the failure
/sqlplan-deadlock — if E22 (login failure burst) or connectivity errors coincide with error
1205 in application logs, analyze the deadlock XML from the system_health XE session
1---2name: errorlog-review3description: Analyzes SQL Server ERRORLOG files for operational issues, availability group failures, memory pressure, I/O subsystem warnings, and security events. Use this skill whenever a SQL Server instance has experienced unexpected behavior, an AG failover, memory warnings, I/O latency alerts, or abnormal shutdown, and you need a structured timeline of what SQL Server recorded. Applies 28 checks (E1–E28) covering AG health, memory/resource pressure, I/O and storage, startup/shutdown, connectivity, and configuration signals.4---5
6# SQL Server ERRORLOG Review Skill
7
8## Purpose
9
10Parse and analyze SQL Server ERRORLOG content to surface operational warnings, high-availability
11failures, resource pressure signals, security events, and configuration anomalies. Applies 28
12checks (E1–E28) across five categories:
13
14- **E1–E8** — AG / High Availability: failovers, lease expiry, replica state changes, synchronization errors
15- **E9–E14** — Memory and resource pressure: page allocation failures, OS paging, worker exhaustion, non-yielding schedulers
16- **E15–E19** — I/O and storage: slow I/O subsystem, corruption warnings, tempdb exhaustion, log backup gaps, VLF proliferation
17- **E20–E24** — Startup, shutdown, and connectivity: abnormal termination, restart cycling, login failure bursts, linked server errors
18- **E25–E28** — Configuration and informational: trace flags, unconfigured max memory, log rotation gaps, version end-of-support
19
20## Input
21
22Accept any of:
23
24- **File path** — path to the SQL Server ERRORLOG file (default location:
25 `C:\Program Files\Microsoft SQL Server\MSSQL<ver>.<inst>\MSSQL\Log\ERRORLOG`)
26- **Inline paste** — raw ERRORLOG text pasted directly into chat; partial excerpts are valid
27- **Natural language description** — describe the symptoms or paste selected log lines with context
28
29For best results, provide the current ERRORLOG and at least one prior log (`ERRORLOG.1`). When
30only partial content is available, state which time range is covered.
31
32### Capture via T-SQL
33
34```sql
35-- Read current ERRORLOG (0 = current, 1 = previous, 2 = the one before that)
36EXEC xp_readerrorlog 0, 1; -- SQL Server log, current file
37EXEC xp_readerrorlog 1, 1; -- SQL Server log, previous file
38
39-- Filter to AG-related messages only
40EXEC xp_readerrorlog 0, 1, N'availability', NULL, NULL, NULL, N'desc';
41
42-- Filter to a time window (last 2 hours)
43DECLARE @start DATETIME = DATEADD(HOUR, -2, GETDATE());
44EXEC xp_readerrorlog 0, 1, NULL, NULL, @start, NULL, N'desc';
45```
46
47### Column Reference
48
49| Column | Meaning |
50|--------|---------|
51| LogDate | Timestamp of the log entry (datetime2 precision) |
52| ProcessInfo | SPID or system process (e.g., `spid28s`, `Logon`, `Backup`) |
53| Text | Log message text |
54
55---
56
57## Thresholds Reference
58
59| Threshold | Value | Used by |
60|-----------|-------|---------|
61| Login failure burst — Warning | > 5 `Login failed` messages in any 5-min window | E22 |
62| Login failure burst — Critical | > 20 `Login failed` messages in any 5-min window | E22 |
63| Restart cycling | ≥ 2 SQL Server startup messages within 60 min | E21 |
64| I/O slow built-in threshold | 15 seconds (SQL Server internal, non-configurable) | E15 |
65| Log backup overdue — FULL/BULK_LOGGED | > 24 hr since last `Database backed up` entry | E18 |
66| Log backup overdue — active log pressure signal | > 8 hr when `log_reuse_wait_desc = LOG_BACKUP` | E18 |
67
68---
69
70## AG / High Availability Checks (E1–E8)
71
72### E1 — AG Failover Event
73- **Trigger:** Log contains `performing a planned role change` or `automatic failover` in the same
74 entry or within the same minute as a role-change message; also `in response to a request from
75 the Windows Server Failover Cluster`
76- **Severity:** Warning — planned failover expected; Critical if the word `automatic` appears
77 (unplanned loss of primary)
78- **Fix:** For unplanned failovers, check E2 (lease expiry) and E6 (health check timeout) as
79 probable root causes. For planned failovers in unexpected windows, review change-management
80 records. Run `/sqlwait-review` on HADR_SYNC_COMMIT and HADR_WORK_QUEUE waits.
81
82### E2 — Lease Expiry
83- **Trigger:** Log contains `lease between the availability group and the Windows Server Failover
84 Cluster has expired` or `The lease of availability group` combined with `has expired`
85- **Severity:** Critical — lease expiry is the most common root cause of unplanned AG failovers
86- **Fix:** Investigate the time immediately before this entry for E15 (slow I/O), E13
87 (non-yielding scheduler), or OS-level events. Common causes: storage latency spike causing
88 the sp_server_diagnostics thread to miss its deadline, high CPU starvation, or WSFC network
89 interruption. Increase `LeaseTimeout` in WSFC only as a temporary measure — fix the root cause.
90
91### E3 — Replica State Change
92- **Trigger:** Log contains `The local replica of availability group ... is changing roles` or
93 `is preparing to transition to the`
94- **Severity:** Warning — state transitions are normal during planned operations; unexpected
95 transitions during business hours warrant investigation
96- **Fix:** Correlate the timestamp with E1 (failover), E2 (lease), or external WSFC events.
97 If unplanned, check the Windows Event Log and WSFC cluster log for the triggering event.
98
99### E4 — AG Database Joining Failure
100- **Trigger:** Log contains `Failed to join local availability database` or `The availability
101 database ... is not in the correct state`
102- **Severity:** Critical — the AG database is not receiving redo; secondary is running but not
103 synchronized, providing false HA coverage
104- **Fix:** Run `SELECT * FROM sys.dm_hadr_database_replica_states` to check
105 `synchronization_state_desc` and `redo_queue_size`. If redo queue is growing, check disk I/O
106 on the secondary. If the database is in `NOT SYNCHRONIZING`, re-join: `ALTER DATABASE [db]
107 SET HADR AVAILABILITY GROUP = [ag_name]`.
108
109### E5 — Data Synchronisation Suspended
110- **Trigger:** Log contains `Synchronization of this database ... has been suspended` or
111 `Data movement for availability database ... has been suspended`
112- **Severity:** Warning — a suspended database is not receiving log records; RPO clock is running
113- **Fix:** Identify whether the suspension was manual (`ALTER DATABASE ... SET HADR SUSPEND`) or
114 automatic (error-triggered). Check for E15/E16 (I/O or corruption) causing automatic suspension.
115 Resume: `ALTER DATABASE [db] SET HADR RESUME`. Monitor redo queue.
116
117### E6 — AG Health Check Timeout
118- **Trigger:** Log contains `availability group ... has failed to take necessary action within
119 the time allotted` or `The availability group ... exceeded the health-check timeout`
120- **Severity:** Critical — health-check failure directly precedes automatic failover; this entry
121 combined with E1 confirms the full failover sequence
122- **Fix:** Identify what the primary was doing at the time. E13 (non-yielding scheduler) or E9
123 (page allocation failure) are common co-occurrences. The `HealthCheckTimeout` WSFC property
124 controls sensitivity — do not increase it without fixing the underlying responsiveness problem.
125
126### E7 — Redo Thread Error
127- **Trigger:** Log contains `An error occurred in the redo thread for database` or
128 `Redo thread for database ... encountered error`
129- **Severity:** Critical — the secondary redo thread has failed; the secondary is no longer
130 applying log records and RPO is accumulating
131- **Fix:** Note the error number in the log message. Common causes: corruption on the secondary
132 (check E16), log record version mismatch after an upgrade, or disk full on secondary. For
133 disk-full, free space and resume synchronization. For corruption, restore the secondary from
134 a backup and re-seed.
135
136### E8 — Secondary Not Synchronising
137- **Trigger:** Log contains `Waiting for redo catch-up` or mentions secondary redo queue in a
138 warning context; or `log send queue` appearing repeatedly with growing values
139- **Severity:** Warning — secondary is lagging; failover to this replica would result in data
140 loss proportional to the redo queue depth
141- **Fix:** Check network bandwidth between primary and secondary. Run
142 `SELECT redo_queue_size, redo_rate FROM sys.dm_hadr_database_replica_states`. If redo rate <
143 log generation rate, the secondary cannot keep up — review disk I/O on secondary (E15) or
144 increase network bandwidth.
145
146---
147
148## Memory and Resource Pressure Checks (E9–E14)
149
150### E9 — FAIL_PAGE_ALLOCATION
151- **Trigger:** Log contains `FAIL_PAGE_ALLOCATION` (exact string, case-insensitive)
152- **Severity:** Critical — SQL Server could not satisfy an internal memory allocation; queries
153 may have failed with out-of-memory errors; this entry often precedes OS paging (E10)
154- **Fix:** Check `max server memory` configuration (E26). Run
155 `SELECT type, pages_kb FROM sys.dm_os_memory_clerks ORDER BY pages_kb DESC` to identify
156 which clerk is consuming the most memory. Consider reducing max server memory by 10–15% to
157 leave headroom for OS and other processes.
158
159### E10 — OS Memory Pressure
160- **Trigger:** Log contains `A significant part of sql server process memory has been paged out`
161 or `Working set trim`
162- **Severity:** Critical — Windows has paged SQL Server memory to disk under OS memory pressure;
163 buffer pool pages are on disk, causing extreme I/O latency
164- **Fix:** Reduce `max server memory` to allow OS headroom (leave at least 10% of RAM or 4 GB,
165 whichever is greater). Enable `Lock Pages in Memory` (LPIM) to prevent paging for 64-bit SQL
166 Server service account. Investigate other processes competing for RAM on the host.
167
168### E11 — Buffer Pool Insufficient
169- **Trigger:** Log contains `There is insufficient system memory in resource pool` or
170 `Memory Manager: Memory node available memory is less than threshold`
171- **Severity:** Critical — queries requiring memory grants are being denied; workload will
172 stall on `RESOURCE_SEMAPHORE` waits
173- **Fix:** Run `/sqlwait-review` and check for `RESOURCE_SEMAPHORE` dominance. Increase
174 `max server memory` if physical RAM allows, or reduce `min memory per query` via Resource
175 Governor. Identify large-grant queries with `/sqlplan-review` S2–S4.
176
177### E12 — Worker Thread Exhaustion
178- **Trigger:** Log contains `There are no more threads available to process new requests` or
179 `Worker Thread ... has been waiting too long`
180- **Severity:** Critical — new connections are being refused or queued; the instance is at
181 maximum worker thread capacity
182- **Fix:** Increase `max worker threads` via `sp_configure` only after identifying root cause.
183 Common causes: blocking chains holding threads (check `sys.dm_exec_requests`), long-running
184 queries, or undersized `max worker threads` for the workload. Run `/sqlwait-review` for
185 `THREADPOOL` waits (V-checks).
186
187### E13 — Scheduler Non-Yielding
188- **Trigger:** Log contains `Process appears to be non-yielding on Scheduler` or
189 `A scheduler appears to be non-yielding`
190- **Severity:** Critical — a thread is monopolising a scheduler without yielding; this blocks
191 all other threads on that scheduler, degrades responsiveness, and can trigger AG health-check
192 timeouts (E6) and lease expiry (E2)
193- **Fix:** A memory dump is typically generated automatically. Look for a `.mdmp` file in the
194 SQL Server Log directory matching the timestamp. Common causes: large in-memory sort, CLR
195 call, XTP operation, or a bug in a specific build — check if a known hotfix applies for the
196 version (E28). Consider enabling `DBCC TRACEON(8086)` on advice from Microsoft Support.
197
198### E14 — Memory Grant Timeout
199- **Trigger:** Log contains `Memory grant request timed out` or
200 `A request for memory failed with OOM (out of memory) status`
201- **Severity:** Warning — a query could not acquire its requested memory grant within the
202 timeout; it may have been killed or retried with a reduced grant, causing a spill to TempDb
203- **Fix:** Capture the affected query and run `/sqlplan-review` for S2–S4 (memory grant checks).
204 Update statistics to improve cardinality estimates. Use Resource Governor to cap grants for
205 ad-hoc workloads. Check for E11 (resource pool exhaustion) as a co-trigger.
206
207---
208
209## I/O and Storage Checks (E15–E19)
210
211### E15 — I/O Subsystem Slow
212- **Trigger:** Log contains `SQL Server has encountered` combined with `I/O requests taking
213 longer than 15 seconds` (SQL Server's built-in slow I/O threshold)
214- **Severity:** Critical — storage latency has exceeded the 15-second internal threshold;
215 this is a primary trigger for AG lease expiry (E2) and health-check timeouts (E6)
216- **Fix:** Note the file path and database in the message. Investigate storage subsystem: check
217 disk queue length, RAID controller cache status, SAN/NVMe latency metrics, and any concurrent
218 backup or maintenance operations competing for I/O. If on a VM, check storage IOPS limits.
219 Run `/sqlwait-review` for `PAGEIOLATCH_SH` and `PAGEIOLATCH_EX` dominance.
220
221### E16 — Database Corruption Warning
222- **Trigger:** Log contains `checksum mismatch`, `torn page`, `consistency errors detected`,
223 or `DBCC CHECKDB found` with error counts > 0
224- **Severity:** Critical — data corruption has been detected; backup integrity is unknown until
225 verified; the affected database may be inaccessible or returning wrong results
226- **Fix:** Run `DBCC CHECKDB ([database]) WITH NO_INFOMSGS` immediately to assess scope. Do
227 not attempt to repair until a current, verified backup exists. For `REPAIR_ALLOW_DATA_LOSS`,
228 treat it as a last resort — restore from backup is always preferable. Investigate E15 (I/O
229 latency) and storage hardware health as root causes.
230
231### E17 — TempDB Space Exhaustion
232- **Trigger:** Log contains `Could not allocate space` combined with `in database 'tempdb'`
233 or `tempdb is full` or `tempdb ran out of space`
234- **Severity:** Critical — queries requiring temporary space (sorts, hashes, spools, row
235 versioning) are failing; error 1105 is returned to applications
236- **Fix:** Immediately: `DBCC SHRINKFILE` on tempdb data files to recover any unused allocated
237 space, or add a tempdb data file. Long term: investigate which query is consuming tempdb
238 (check `sys.dm_db_session_space_usage`). Run `/sqlplan-review` for N41–N43 (spill operators).
239 Consider pre-allocating tempdb to expected working size at startup.
240
241### E18 — Log Backup Overdue
242- **Trigger:** Gap between consecutive `Database backed up` entries for the same database
243 exceeds the threshold for that recovery model. For databases in FULL or BULK_LOGGED recovery,
244 flag if the gap exceeds 24 hours; flag more urgently if log backup entries are absent while
245 other evidence suggests active transaction log growth
246- **Severity:** Warning — log space will grow unboundedly without log backups; in a FULL
247 recovery database, the log cannot be truncated until backed up
248- **Fix:** Run a log backup immediately: `BACKUP LOG [database] TO DISK = N'path\logbackup.bak'`.
249 Verify the SQL Agent log backup job is scheduled and enabled. Check `sys.databases` column
250 `log_reuse_wait_desc` — if `LOG_BACKUP`, the log is waiting for a backup to allow truncation.
251
252### E19 — VLF Proliferation Signal
253- **Trigger:** Log shows repeated `autogrow` events on transaction log files (multiple autogrow
254 completions in the log window), or the database log has grown significantly between ERRORLOG
255 entries — inferred from repeated log file path growth messages
256- **Severity:** Info — excessive VLFs degrade recovery time and log-backup performance; auto-grow
257 events indicate the log was not sized for the workload
258- **Fix:** Shrink and pre-size the log: set the initial log file size to cover expected working
259 set and disable autogrow on the log (or set a large, infrequent growth increment). Run
260 `DBCC LOGINFO ([database])` to count current VLFs — if > 1,000, shrink and re-expand in one
261 step. Align with E18 (log backup cadence) to ensure the log truncates regularly.
262
263---
264
265## Startup, Shutdown, and Connectivity Checks (E20–E24)
266
267### E20 — Abnormal Shutdown
268- **Trigger:** Log contains `SQL Server is terminating` or `SQL Server has encountered` combined
269 with `stack dump` or shutdown messages, without a preceding graceful shutdown marker
270 (`SQL Server is terminating due to a system shutdown request` at the end of the prior log file)
271- **Severity:** Critical — the instance crashed rather than shut down cleanly; uncommitted
272 transactions were rolled back on restart; any in-flight work is lost
273- **Fix:** Check the Windows Event Log (`Application` and `System` sources) for the crash
274 timestamp. Look for a dump file in the SQL Server Log directory. If the crash occurred
275 mid-transaction in an AG, check whether secondary databases advanced beyond the primary
276 (split-brain risk). Engage Microsoft Support with the minidump if the crash is reproducible.
277
278### E21 — Repeated Restarts
279- **Trigger:** ERRORLOG or combined ERRORLOG + ERRORLOG.1 contains ≥ 2 SQL Server startup
280 messages (lines containing `SQL Server is starting` or `This instance of SQL Server last
281 reported using a process ID`) within a 60-minute window
282- **Severity:** Critical — the instance is crash-looping; each restart drops all plan cache and
283 connection state; applications experience repeated connection failures
284- **Fix:** Check E20 (abnormal shutdown) for the crash cause between restarts. If the instance
285 is restarting due to a failed startup condition (e.g., tempdb creation failure, master database
286 corruption, or xp_cmdshell misconfiguration), resolve the startup error first. Enable Windows
287 `Automatic Recovery` only after identifying the underlying fault.
288
289### E22 — Login Failure Burst
290- **Trigger:** Count of `Login failed` entries exceeds the threshold within a 5-minute rolling
291 window — see Thresholds Reference for Warning and Critical levels
292- **Severity:** Warning if > 5 failures in 5 min; Critical if > 20 failures in 5 min
293- **Fix:** Identify the `ClientConnectionID` and source IP in the failure messages. A burst from
294 one account likely indicates a misconfigured application connection string after a password
295 rotation. A burst from many accounts may indicate a brute-force attempt. For brute-force:
296 enable SQL Server Audit or Extended Events on `Failed Logins` and block the source IP at the
297 network layer. Ensure `LOGINAUDIT` is set to `Failed logins only` or `Both` in Server
298 properties so future bursts appear in the ERRORLOG.
299
300### E23 — Linked Server Error
301- **Trigger:** Log contains `OLE DB provider` combined with `reported an error` or
302 `Cannot obtain the required interface` for a linked server provider
303- **Severity:** Warning — distributed queries or cross-server stored procedures using this
304 linked server will fail until the provider error is resolved
305- **Fix:** Identify the linked server name and provider from the error text. Common causes:
306 target server unavailable, credential expiry, or OLE DB provider version mismatch. Test
307 connectivity: `EXEC sp_testlinkedserver [linked_server_name]`. If the provider is outdated,
308 update it on the SQL Server host.
309
310### E24 — Connectivity Error
311- **Trigger:** Log contains `A connection was successfully established with the server, but
312 then an error occurred during the login process` or `The connection has been lost` or
313 `A network-related or instance-specific error` in the ERRORLOG (as opposed to the client)
314- **Severity:** Warning — SQL Server is logging errors from its own outbound connections
315 (linked servers, distributed queries, SSISDB, mail, replication) or from incoming connections
316 that dropped after TCP establishment
317- **Fix:** Correlate the timestamp with E22 (login failures), network infrastructure changes,
318 or TLS/SSL certificate renewals. If `TLS handshake` appears in the message, verify that
319 the certificate in use has not expired and that the client supports the negotiated protocol.
320
321---
322
323## Configuration and Informational Checks (E25–E28)
324
325### E25 — Trace Flag Active
326- **Trigger:** Log contains `Trace flag` combined with `is set` or `was enabled at startup`
327 in startup messages
328- **Severity:** Info — trace flags change engine behavior; document intent and verify they
329 are still appropriate for the current SQL Server version
330- **Fix:** List all active trace flags: `DBCC TRACESTATUS(-1)`. Common production trace flags
331 and their intent: 1117/1118 (tempdb allocation — superseded in 2016+), 3226 (suppress
332 successful backup log entries), 4199 (QO hotfixes). Remove trace flags that are no longer
333 needed or that apply to behaviour fixed in a later CU.
334
335### E26 — Max Server Memory Default
336- **Trigger:** Log contains startup line showing `max server memory` = 2147483647 MB, or the
337 instance has been running with the default (unlimited) memory configuration — inferred from
338 startup messages or the absence of an explicit `max server memory` setting entry
339- **Severity:** Info — unlimited memory allows SQL Server to consume all available RAM, starving
340 the OS and any other services, which can trigger E10 (OS paging)
341- **Fix:** Set `max server memory` to total RAM minus OS headroom: leave at least 10% of RAM or
342 4 GB (whichever is larger) for the OS. For example, on a 64 GB server:
343 `EXEC sp_configure 'max server memory (MB)', 57344; RECONFIGURE;`
344
345### E27 — ERRORLOG Rotation Gap
346- **Trigger:** Only a single ERRORLOG file is provided, covering a window shorter than 24 hours,
347 with no prior context from ERRORLOG.1 or earlier
348- **Severity:** Info — events before the current file (including the original startup, prior
349 AG failovers, or earlier memory events) are not visible; findings may be incomplete
350- **Fix:** Retrieve prior ERRORLOG files: `EXEC xp_readerrorlog 1, 1` through
351 `EXEC xp_readerrorlog 6, 1` (SQL Server retains up to 6 prior logs by default, configurable
352 in SSMS → Server Properties → Database Settings → Number of error log files). State in the
353 report: "Analysis covers [start] – [end] only; prior events not available."
354
355### E28 — SQL Server Version
356- **Trigger:** Startup line containing `Microsoft SQL Server 20XX` version string — present
357 in every ERRORLOG at instance start
358- **Severity:** Info — extract and evaluate: (1) is this build on extended support, mainstream
359 support, or past end-of-support? (2) is this the latest CU for this major version?
360- **Fix:** Compare the build number in the log against the Microsoft SQL Server build list.
361 If past end-of-support (e.g., SQL 2014 EOL 2019-07-09, SQL 2016 EOL 2026-07-14), plan
362 upgrade. If not on the latest CU, evaluate whether open bugs fixed in later CUs are relevant
363 to the observed issues. Report the version string verbatim in the Output Summary.
364
365---
366
367## Output Format
368
369Structure the report as follows. Use this exact section order.
370
371```
372## SQL Server ERRORLOG Analysis
373
374### Summary
375- X Critical, Y Warnings, Z Info
376- Time range: [first log entry datetime] – [last log entry datetime]
377- SQL Server version: [version string from E28 startup line, or "Not found in provided excerpt"]
378- Highest-risk finding: [check name and ID, e.g., "E2 — Lease Expiry"]
379- Log coverage note: [single file / multiple files / partial excerpt — dates if known]
380
381### Critical Issues
382
383### [C1 — E2] Lease Expiry (2026-01-15 14:32:05)
384- **Observed:** "lease between the availability group 'AG1' and the Windows Server Failover
385 Cluster has expired" at 14:32:05. Preceded by E15 (I/O slow) at 14:28:44 on
386 E:\Data\AG1_Primary.mdf.
387- **Impact:** Unplanned AG failover triggered. AG1 primary role transferred to secondary.
388 Applications lost primary connection for the duration of the failover.
389- **Fix:** Investigate I/O latency on E:\Data at 14:28 (see C2 — E15). Do not increase
390 LeaseTimeout without resolving the root cause I/O delay.
391
392### Warnings
393
394### [W1 — E1] AG Failover Event (2026-01-15 14:32:08)
395...
396
397### Info
398
399### [I1 — E25] Trace Flag Active (startup)
400...
401
402### Passed Checks
403
404| Check | Result |
405|-------|--------|
406| E9 — FAIL_PAGE_ALLOCATION | PASS — no FAIL_PAGE_ALLOCATION entries found |
407| E16 — Database Corruption Warning | PASS — no checksum or torn-page errors found |
408```
409
410Each finding label uses `[C1]`, `[W1]`, `[I1]` sequence numbering, with the check ID in
411parentheses. Findings reference related checks by ID where one explains another
412(e.g., "root cause of C1 — E2"). Passed Checks must list every check explicitly evaluated.
413When a check cannot be evaluated (e.g., E18 with no backup log entries), state
414"SKIP — no `Database backed up` entries in provided log window" rather than PASS or FAIL.
415
416---
417
418## Notes
419
420- ERRORLOG entries use local server time — note timezone if it differs from the analyst's context.
421- Messages from `spid28s` (or any `spidNs`) are system threads; `Logon` is the login auditing
422 thread; `Backup` is the backup thread.
423- When multiple ERRORLOG files span a long window, the startup entry in each file signals the
424 beginning of a new SQL Server process (i.e., a restart occurred between files).
425- The ERRORLOG does not record all events — OS-level events (WSFC partitions, disk controller
426 errors) appear only in the Windows Event Log and WSFC cluster log. Reference the companion
427 skill list below for those artifacts when ERRORLOG evidence points to external causes.
428- Do not report a PASS for E18 if no `Database backed up` entries are present — the absence
429 of backup log entries is itself an E18 signal for databases in FULL recovery. State clearly
430 which databases had backup evidence and which did not.
431
432## Companion Skills
433
434- `/sqlwait-review` — correlate ERRORLOG memory and I/O signals (E9–E15) with
435 `PAGEIOLATCH_SH`, `RESOURCE_SEMAPHORE`, `HADR_SYNC_COMMIT`, and `THREADPOOL` wait dominance
436- `/sqlplan-review` + `/sqlplan-index-advisor` — analyze execution plans for queries that were
437 running during the incident window; high-cost queries during a memory or I/O event often
438 accelerate the failure
439- `/query-store-review` — identify plan regressions introduced after a post-incident restart
440 clears the plan cache, causing previously stable queries to recompile with bad plans
441- `/tsql-review` — review T-SQL source of stored procedures flagged during the incident as
442 high resource consumers before and after the failure
443- `/sqlplan-deadlock` — if E22 (login failure burst) or connectivity errors coincide with error
444 1205 in application logs, analyze the deadlock XML from the `system_health` XE session