Query Optimization
Comprehensive guide to T-SQL query optimization for SQL Server and Azure SQL Database. Optimize from verified evidence: schema, data types, indexes, row counts, partitioning, execution plans, and allowed change types.
Mandatory Workflow
1. Schema-First Validation
Before recommending rewrites, indexes, or hints, verify or mark unknown:
- SQL Server version, edition, compatibility level, and Azure SQL tier if applicable.
- Full query/procedure text, parameter values, frequency, and runtime target.
- DDL for tables, views, temp tables, table variables, and TVFs.
- Data types for join, filter, grouping, ordering, and temp-table columns, including length, precision, scale, collation, and nullability.
- Existing indexes, constraints, statistics age, partition function/scheme, and row counts.
- Whether objects are local or linked-server/remote sources.
- Allowed change types: query rewrite, new index, huge-table index change, stats update, staging table, computed column, or no code change.
Use ../_shared/optimization-intake.md and ../_shared/assumption-tracker.md. If key facts are missing, provide conditional guidance plus diagnostics instead of final prescriptions.
2. Identify the Root Bottleneck
Use actual execution plans and STATISTICS IO, TIME when possible. Rank findings by measured impact, not just estimated plan percentage:
- High logical reads or row reads.
- Bad estimate vs actual row gaps.
- Scans caused by non-SARGable predicates or missing access paths.
- Key lookups multiplied by many executions.
- Sort/hash/window spills from poor estimates or missing order.
- Remote queries that fail to push predicates or joins to the linked server.
For detailed .sqlplan inspection, load tsql-master:execution-plan-analysis.
3. Fix SARGability and Type Mismatches
SARGable predicates can use ordered index access. Avoid functions or conversions on indexed columns.
| Non-SARGable |
Safer pattern |
WHERE YEAR(OrderDate) = 2026 |
WHERE OrderDate >= '20260101' AND OrderDate < '20270101' |
WHERE LEFT(Name, 3) = 'ABC' |
WHERE Name LIKE 'ABC%' |
WHERE Amount * 1.1 > 1000 |
WHERE Amount > 1000 / 1.1 |
WHERE CONVERT(date, Dt) = @d |
WHERE Dt >= @d AND Dt < DATEADD(day, 1, @d) |
WHERE VarcharCol = 123 |
WHERE VarcharCol = '123' |
Check actual data types. A syntactically SARGable predicate can still scan if a parameter, temp column, or join key has the wrong type or collation.
4. Prove Join Changes
Never remove or replace joins only because selected columns come from one table. Prove:
- Does the join filter rows? Compare base count vs joined count and check trusted foreign keys.
- Does the join multiply rows? Check uniqueness on the joined key and duplicates in the referenced table.
- Can it be replaced with
EXISTS? Use a semi-join when only existence is needed and row multiplication must be avoided.
- Are outer joins preserved? Predicates in
WHERE can accidentally convert LEFT JOIN to inner join.
Move join-removal experiments into a proof harness and validate equivalence with EXCEPT in both directions. See references/rewrite-proof-harnesses.md.
5. Check Temp Tables and Staging Types
Temp tables are often the right optimization tool, but bad types can create hidden conversions.
Before using or recommending a temp table:
- Compare staged column types against source metadata.
- Match string length and collation for join/filter columns.
- Match numeric precision/scale and date/time precision.
- Add appropriate clustered or nonclustered indexes after load when row counts justify them.
- Update temp-table statistics when phased optimization depends on accurate cardinality.
Flag mismatches as first-order findings because they can invalidate plan analysis and index recommendations. Use the checker in references/rewrite-proof-harnesses.md.
6. Select the Rewrite Template
Choose the least invasive rewrite that addresses the verified bottleneck:
| Situation |
Template |
| Highly selective predicate before huge joins |
Stage selective keys first, index the stage, then join. |
| Huge detail table aggregated later |
Aggregate early if grouping preserves semantics. |
| Join only tests existence |
Replace row-producing join with EXISTS. |
| OR across different columns |
Split into UNION ALL branches with duplicate guards. |
| Catch-all optional predicates |
Dynamic SQL or targeted recompilation; avoid Col = @p OR @p IS NULL for hot paths. |
| Unsafe partition predicate |
Rewrite to direct typed range on partition column. |
| Bad estimates from table variables/TVFs |
Use temp tables, inline TVFs, or recompile depending on version and workload. |
Do not stage huge unfiltered tables or aggregate early unless the row reduction and semantic equivalence are proven.
Parameter Sensitivity
Parameter sniffing occurs when a plan compiled for one value is reused for a very different value. Confirm skew and compile/runtime values before applying fixes.
| Option |
Best for |
Caution |
OPTION (RECOMPILE) |
Infrequent or highly variable statements |
Adds compile CPU; plan not reused. |
OPTIMIZE FOR (@p = value) |
Stable representative value |
Can age badly as data changes. |
OPTIMIZE FOR UNKNOWN |
Average distribution is acceptable |
Can be mediocre for all cases. |
| Dynamic SQL |
Optional predicates and varied shapes |
Requires safe parameterization. |
| Query Store hints |
SQL Server 2022+ or Azure SQL, no code change |
Monitor regressions. |
| PSP optimization |
SQL Server 2022+ with compatibility 160 |
Only applies to eligible patterns. |
Execution Plan Checks
Watch these operators and warnings:
| Plan evidence |
Likely action |
| Scan with residual predicate |
Fix SARGability, key order, or filtered index. |
| Seek with high rows read |
Add more selective key columns or rewrite residual predicate. |
| Key lookup repeated many times |
Cover query or reduce outer rows first. |
| Sort spill or hash spill |
Fix estimates, reduce rows/width, add order-compatible index. |
CONVERT_IMPLICIT on column |
Align parameter/temp/source data types. |
| Estimate off by 10x+ |
Check stats, skew, table variables, predicates, constraints. |
| Missing-index warning |
Treat as candidate only; merge with existing indexes and workload. |
Statistics and Cardinality
Use statistics work when evidence points to stale or insufficient estimates:
DBCC SHOW_STATISTICS('dbo.TableName', 'IndexOrStatsName');
UPDATE STATISTICS dbo.TableName IndexOrStatsName WITH FULLSCAN;
For large partitioned tables, evaluate incremental statistics and filtered stats. Do not run broad fullscan updates in production without maintenance-window and blocking considerations.
Output Format
Respond with:
- Intake status: verified, unverified, disproved, needs diagnostic.
- Bottleneck evidence: plan nodes, reads, rows, estimates, warnings.
- Recommendation path: rewrite, index/stat change, or diagnostic, separated by allowed change type.
- Proof harness: result equivalence and before/after performance metrics.
- Risks: parameter sensitivity, write overhead, blocking, partition safety, remote pushdown.
References
../_shared/optimization-intake.md - mandatory intake checklist.../_shared/assumption-tracker.md - assumption status protocol.references/rewrite-proof-harnesses.md - join proof, temp type checker, and rewrite templates.references/dmv-diagnostic-queries.md - DMV queries for performance analysis.
1---2name: query-optimization3description: This skill should be used when the user asks to optimize slow T-SQL queries, fix SQL Server performance regressions, improve SARGability, rewrite joins, prove whether joins can be removed, compare temp-table data types, select rewrite templates, diagnose parameter sniffing, evaluate query hints, inspect statistics/cardinality estimates, reduce spills or memory grants, use Query Store, or interpret plan evidence. PROACTIVELY activate for slow queries, scans vs seeks, implicit conversions, bad joins, tempdb spills, query rewrites, partition predicate safety, and before recommending indexes or hints. Provides: evidence-first workflow, rewrite templates, parameter-sniffing playbook, and gating rules before recommending indexes or hints.4---5
6# Query Optimization
7
8Comprehensive guide to T-SQL query optimization for SQL Server and Azure SQL Database. Optimize from verified evidence: schema, data types, indexes, row counts, partitioning, execution plans, and allowed change types.
9
10## Mandatory Workflow
11
12### 1. Schema-First Validation
13
14Before recommending rewrites, indexes, or hints, verify or mark unknown:
15
16- SQL Server version, edition, compatibility level, and Azure SQL tier if applicable.
17- Full query/procedure text, parameter values, frequency, and runtime target.
18- DDL for tables, views, temp tables, table variables, and TVFs.
19- Data types for join, filter, grouping, ordering, and temp-table columns, including length, precision, scale, collation, and nullability.
20- Existing indexes, constraints, statistics age, partition function/scheme, and row counts.
21- Whether objects are local or linked-server/remote sources.
22- Allowed change types: query rewrite, new index, huge-table index change, stats update, staging table, computed column, or no code change.
23
24Use `../_shared/optimization-intake.md` and `../_shared/assumption-tracker.md`. If key facts are missing, provide conditional guidance plus diagnostics instead of final prescriptions.
25
26### 2. Identify the Root Bottleneck
27
28Use actual execution plans and `STATISTICS IO, TIME` when possible. Rank findings by measured impact, not just estimated plan percentage:
29
30- High logical reads or row reads.
31- Bad estimate vs actual row gaps.
32- Scans caused by non-SARGable predicates or missing access paths.
33- Key lookups multiplied by many executions.
34- Sort/hash/window spills from poor estimates or missing order.
35- Remote queries that fail to push predicates or joins to the linked server.
36
37For detailed `.sqlplan` inspection, load `tsql-master:execution-plan-analysis`.
38
39### 3. Fix SARGability and Type Mismatches
40
41SARGable predicates can use ordered index access. Avoid functions or conversions on indexed columns.
42
43| Non-SARGable | Safer pattern |
44|---|---|
45| `WHERE YEAR(OrderDate) = 2026` | `WHERE OrderDate >= '20260101' AND OrderDate < '20270101'` |
46| `WHERE LEFT(Name, 3) = 'ABC'` | `WHERE Name LIKE 'ABC%'` |
47| `WHERE Amount * 1.1 > 1000` | `WHERE Amount > 1000 / 1.1` |
48| `WHERE CONVERT(date, Dt) = @d` | `WHERE Dt >= @d AND Dt < DATEADD(day, 1, @d)` |
49| `WHERE VarcharCol = 123` | `WHERE VarcharCol = '123'` |
50
51Check actual data types. A syntactically SARGable predicate can still scan if a parameter, temp column, or join key has the wrong type or collation.
52
53### 4. Prove Join Changes
54
55Never remove or replace joins only because selected columns come from one table. Prove:
56
57- **Does the join filter rows?** Compare base count vs joined count and check trusted foreign keys.
58- **Does the join multiply rows?** Check uniqueness on the joined key and duplicates in the referenced table.
59- **Can it be replaced with `EXISTS`?** Use a semi-join when only existence is needed and row multiplication must be avoided.
60- **Are outer joins preserved?** Predicates in `WHERE` can accidentally convert `LEFT JOIN` to inner join.
61
62Move join-removal experiments into a proof harness and validate equivalence with `EXCEPT` in both directions. See `references/rewrite-proof-harnesses.md`.
63
64### 5. Check Temp Tables and Staging Types
65
66Temp tables are often the right optimization tool, but bad types can create hidden conversions.
67
68Before using or recommending a temp table:
69
70- Compare staged column types against source metadata.
71- Match string length and collation for join/filter columns.
72- Match numeric precision/scale and date/time precision.
73- Add appropriate clustered or nonclustered indexes after load when row counts justify them.
74- Update temp-table statistics when phased optimization depends on accurate cardinality.
75
76Flag mismatches as first-order findings because they can invalidate plan analysis and index recommendations. Use the checker in `references/rewrite-proof-harnesses.md`.
77
78### 6. Select the Rewrite Template
79
80Choose the least invasive rewrite that addresses the verified bottleneck:
81
82| Situation | Template |
83|---|---|
84| Highly selective predicate before huge joins | Stage selective keys first, index the stage, then join. |
85| Huge detail table aggregated later | Aggregate early if grouping preserves semantics. |
86| Join only tests existence | Replace row-producing join with `EXISTS`. |
87| OR across different columns | Split into `UNION ALL` branches with duplicate guards. |
88| Catch-all optional predicates | Dynamic SQL or targeted recompilation; avoid `Col = @p OR @p IS NULL` for hot paths. |
89| Unsafe partition predicate | Rewrite to direct typed range on partition column. |
90| Bad estimates from table variables/TVFs | Use temp tables, inline TVFs, or recompile depending on version and workload. |
91
92Do not stage huge unfiltered tables or aggregate early unless the row reduction and semantic equivalence are proven.
93
94## Parameter Sensitivity
95
96Parameter sniffing occurs when a plan compiled for one value is reused for a very different value. Confirm skew and compile/runtime values before applying fixes.
97
98| Option | Best for | Caution |
99|---|---|---|
100| `OPTION (RECOMPILE)` | Infrequent or highly variable statements | Adds compile CPU; plan not reused. |
101| `OPTIMIZE FOR (@p = value)` | Stable representative value | Can age badly as data changes. |
102| `OPTIMIZE FOR UNKNOWN` | Average distribution is acceptable | Can be mediocre for all cases. |
103| Dynamic SQL | Optional predicates and varied shapes | Requires safe parameterization. |
104| Query Store hints | SQL Server 2022+ or Azure SQL, no code change | Monitor regressions. |
105| PSP optimization | SQL Server 2022+ with compatibility 160 | Only applies to eligible patterns. |
106
107## Execution Plan Checks
108
109Watch these operators and warnings:
110
111| Plan evidence | Likely action |
112|---|---|
113| Scan with residual predicate | Fix SARGability, key order, or filtered index. |
114| Seek with high rows read | Add more selective key columns or rewrite residual predicate. |
115| Key lookup repeated many times | Cover query or reduce outer rows first. |
116| Sort spill or hash spill | Fix estimates, reduce rows/width, add order-compatible index. |
117| `CONVERT_IMPLICIT` on column | Align parameter/temp/source data types. |
118| Estimate off by 10x+ | Check stats, skew, table variables, predicates, constraints. |
119| Missing-index warning | Treat as candidate only; merge with existing indexes and workload. |
120
121## Statistics and Cardinality
122
123Use statistics work when evidence points to stale or insufficient estimates:
124
125```sql
126DBCC SHOW_STATISTICS('dbo.TableName', 'IndexOrStatsName');
127UPDATE STATISTICS dbo.TableName IndexOrStatsName WITH FULLSCAN;
128```
129
130For large partitioned tables, evaluate incremental statistics and filtered stats. Do not run broad fullscan updates in production without maintenance-window and blocking considerations.
131
132## Output Format
133
134Respond with:
135
1361. **Intake status**: verified, unverified, disproved, needs diagnostic.
1372. **Bottleneck evidence**: plan nodes, reads, rows, estimates, warnings.
1383. **Recommendation path**: rewrite, index/stat change, or diagnostic, separated by allowed change type.
1394. **Proof harness**: result equivalence and before/after performance metrics.
1405. **Risks**: parameter sensitivity, write overhead, blocking, partition safety, remote pushdown.
141
142## References
143
144- `../_shared/optimization-intake.md` - mandatory intake checklist.
145- `../_shared/assumption-tracker.md` - assumption status protocol.
146- `references/rewrite-proof-harnesses.md` - join proof, temp type checker, and rewrite templates.
147- `references/dmv-diagnostic-queries.md` - DMV queries for performance analysis.
josiahsiegel