How do I install the Conformance Suite skill?

Run npx skillmds@latest add devolutions/conformance-suite in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

Is Conformance Suite safe to use?

This skill has not completed SkillMD's automated safety review yet. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Conformance Suite?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Conformance Suite free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

Who published Conformance Suite?

devolutions (@devolutions) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Conformance Suite by devolutions — Agent Skill

Name: Conformance Suite
Author: devolutions

Conformance Suite

How the SQLite conformance corpus (`conformance/sqlite-sqltests/`) and the expected-failures file work, and how to close a conformance gap correctly. Use this when a sqltest fails or when closing an engine gap.

devolutions ac23282 2.2 KB Updated Sep 17, 2026

File contents

Conformance suite

conformance/sqlite-sqltests/ is a vendored, read-only corpus of 270 .sqltest files (~8,163 cases) that exercises the SQLite SQL surface. It is the primary correctness signal that Ahtola matches SQLite/Turso behavior.

Hard rules

Do not edit .sqltest files to fix tests. Fix the engine. The corpus is upstream; editing it silently weakens correctness.
Conformance gaps belong in the expected-failures file, not in [Ignore] attributes scattered across tests. Don't sprinkle [Ignore] to hide a gap.
When you close a gap, remove the corresponding expected-failure line — do not leave a passing case listed as an expected failure.

Runner

The runner lives in src/Ahtola.Tests/Sqltest/: SqltestParser (parses @database, @skip, test NAME { … } expect { … }), SqltestCorpus (loads the 270 files), SqltestManagedRunner (executes against the managed engine).

Expected-failures file

src/Ahtola.Tests/Conformance/managed-sqltest-expected-failures.txt — one line per known gap, format <file>::<test> | <summary> (see the header comment in the file itself). When the engine does not yet satisfy a case, add a line here with a short summary of why. When the engine catches up, delete the line.

Regenerating

Use the RegenerateExpectedFailures filter to regenerate the file from a full run — but only after you have actually fixed or intentionally accepted the gaps. Review the diff before committing; a regenerated file that "accidentally" drops a still-failing case hides a regression.

Closing a gap

Run the specific case: pwsh ./scripts/Invoke-ManagedTestSuite.ps1 -Framework net10.0 -Filter "FullyQualifiedName~<case>" -MinimumExecutedTests 1.
Fix the engine in Ahtola.Core (consult turso-src/ for the upstream semantics — see the turso-source-ref skill).
Re-run the full conformance lane to confirm you didn't regress another case.
Remove the line from managed-sqltest-expected-failures.txt.

---name: conformance-suitedescription: How the SQLite conformance corpus (`conformance/sqlite-sqltests/`) and the expected-failures file work, and how to close a conformance gap correctly. Use this when a sqltest fails or when closing an engine gap.---# Conformance suite`conformance/sqlite-sqltests/` is a vendored, **read-only** corpus of 270`.sqltest` files (~8,163 cases) that exercises the SQLite SQL surface. It isthe primary correctness signal that Ahtola matches SQLite/Turso behavior.## Hard rules- **Do not edit `.sqltest` files to fix tests.** Fix the engine. The corpus is upstream; editing it silently weakens correctness.- Conformance gaps belong in the expected-failures file, **not** in `[Ignore]` attributes scattered across tests. Don't sprinkle `[Ignore]` to hide a gap.- When you close a gap, **remove the corresponding expected-failure line** — do not leave a passing case listed as an expected failure.## RunnerThe runner lives in `src/Ahtola.Tests/Sqltest/`:`SqltestParser` (parses `@database`, `@skip`, `test NAME { … } expect { … }`),`SqltestCorpus` (loads the 270 files), `SqltestManagedRunner` (executes againstthe managed engine).## Expected-failures file`src/Ahtola.Tests/Conformance/managed-sqltest-expected-failures.txt` — oneline per known gap, format `<file>::<test> | <summary>` (see the headercomment in the file itself). When the engine does not yet satisfy a case, adda line here with a short summary of why. When the engine catches up, deletethe line.## RegeneratingUse the `RegenerateExpectedFailures` filter to regenerate the file from a fullrun — but only after you have actually fixed or intentionally accepted thegaps. Review the diff before committing; a regenerated file that "accidentally"drops a still-failing case hides a regression.## Closing a gap1. Run the specific case: `pwsh ./scripts/Invoke-ManagedTestSuite.ps1 -Framework net10.0 -Filter "FullyQualifiedName~<case>" -MinimumExecutedTests 1`.2. Fix the engine in `Ahtola.Core` (consult `turso-src/` for the upstream semantics — see the `turso-source-ref` skill).3. Re-run the full conformance lane to confirm you didn't regress another case.4. Remove the line from `managed-sqltest-expected-failures.txt`.

Frequently asked questions