Upgrade Codeql CLI And Packs

Upgrade CodeQL CLI and QL Packs

This skill guides you through upgrading the CodeQL CLI version used by the MCP server and synchronizing all ql-mcp-* query packs to use compatible library versions.

When to Use This Skill

• A new CodeQL CLI version has been released
• You need to update .codeql-version to a newer version
• Query unit tests are failing with database compatibility errors
• You see errors like "Cannot downgrade to..." or "database is not compatible with a QL library"

Prerequisites

• Access to the CodeQL CLI (via gh codeql extension or direct installation)

Key Concepts

Version Alignment Strategy

This repository uses a CLI-aligned versioning strategy across all version-bearing files:

1. .codeql-version: Contains the target CLI version (e.g., vX.Y.Z)
2. package.json versions: All package.json files (root, client, extensions/vscode, server) use the CLI version number without the "v" prefix (e.g., X.Y.Z)
3. ql-mcp-* pack versions: Use the CLI version number without the "v" prefix (e.g., X.Y.Z)
4. codeql/*-all dependencies: Must have cliVersion <= target CLI version

Why Database Compatibility Matters

CodeQL databases contain a dbscheme that defines the database structure. The dbscheme in:

• The CLI's extractor (used when creating databases)
• The codeql/*-all library pack (used when running queries)

Must match exactly, or queries will fail with compatibility errors.

Common Compatibility Error

A fatal error occurred: The CodeQL database is not compatible with a QL library referenced by the query you are trying to run.
The database may be too new for the QL libraries the query is using; try upgrading them.

This occurs when:

• The CLI extractor uses a newer dbscheme than the library pack expects
• Using wildcard (*) dependencies that resolve to incompatible pack versions

Upgrade Workflow

Phase 1: Update CLI Version

1.1 Update .codeql-version

Edit .codeql-version to the new target version:

vX.XX.Y

1.2 Install the New CLI Version

gh codeql set-version vX.XX.Y
codeql version  # Verify installation

1.3 Update All Version-Bearing Files

Use the update-release-version.sh script to deterministically update .codeql-version, all package.json files, and all codeql-pack.yml files in a single command:

./server/scripts/update-release-version.sh X.XX.Y

This updates all version-bearing files. Preview changes first with --dry-run:

./server/scripts/update-release-version.sh --dry-run X.XX.Y

Verify consistency with --check:

./server/scripts/update-release-version.sh --check X.XX.Y

After updating, regenerate the lock file:

npm install

Phase 2: Identify Compatible Pack Versions

2.1 List Available Packs

Use the codeql_pack_ls MCP tool to see what pack versions are installed:

{
  "dir": "~/.codeql/packages",
  "format": "json"
}

2.2 Check Pack CLI Compatibility

For each codeql/*-all pack, verify it was built for a compatible CLI version by checking the cliVersion field in its qlpack.yml:

for lang in actions cpp csharp go java javascript python ruby rust swift; do
  version=$(ls ~/.codeql/packages/codeql/${lang}-all/ | head -1)
  echo "$lang-all@$version: $(cat ~/.codeql/packages/codeql/${lang}-all/$version/qlpack.yml | grep cliVersion)"
done

Important: The pack's cliVersion must be ≤ your target CLI version.

2.3 Download Older Compatible Versions (If Needed)

If a pack has cliVersion newer than your target CLI, download an older version:

gh codeql pack download "codeql/{language}-all@{older-version}"

Then re-verify the cliVersion is compatible.

Phase 3: Update codeql-pack.yml Files

Note: The version field in all codeql-pack.yml files is already updated by the update-release-version.sh script in Phase 1.3. This phase focuses on updating codeql/*-all dependency versions for compatibility.

3.1 Files to Update

All codeql-pack.yml files under server/ql/*/tools/:

3.2 Source Pack Template

name: advanced-security/ql-mcp-{language}-tools-src
version: X.XX.Y # CLI version without 'v' prefix
library: false
dependencies:
  codeql/{language}-all: { compatible-version } # Explicit version, no wildcards

3.3 Test Pack Template

name: advanced-security/ql-mcp-{language}-tools-test
version: X.XX.Y # CLI version without 'v' prefix
dependencies:
  advanced-security/ql-mcp-{language}-tools-src: ${workspace}
extractor: { language }

Phase 4: Regenerate Lock Files

Use the codeql_pack_install MCP tool for each pack directory, or run the helper script:

./server/scripts/install-packs.sh

Or use the codeql_pack_install MCP tool for individual packs:

{
  "packDir": "server/ql/{language}/tools/src"
}

Phase 5: Validate Changes

5.1 Run Query Unit Tests

Use the codeql_test_run MCP tool:

{
  "tests": ["server/ql/{language}/tools/test"]
}

Or run all tests with the helper script:

./server/scripts/run-query-unit-tests.sh

5.2 Handle Test Output Differences

Some library version upgrades cause non-deterministic output ordering changes. If tests fail with only ordering differences in .expected files (not logic changes), update the expected files to match the new output.

Quick Reference: Languages and Pack Names

Lessons Learned

Never Use Wildcards in Dependencies

Wildcards resolve to the latest version, which may be incompatible:

# Bad - may resolve to incompatible version
dependencies:
  codeql/cpp-all: '*'

# Good - explicit compatible version
dependencies:
  codeql/cpp-all: 1.2.3

Pack cliVersion Rules

• Pack cliVersion must be ≤ target CLI version
• Packs built for the same minor version (e.g., X.Y.x) are usually compatible
• Different languages may require different pack versions due to independent release cycles

Test Output Changes

Library upgrades can cause non-deterministic ordering changes in query output. These are cosmetic differences - update .expected files when the logic is unchanged but output order differs.

npm Package files Field Limitations

npm's files field does not support intermediate wildcard patterns like ql/*/tools/src/. Each language directory must be listed explicitly:

"files": [
  "dist/",
  "ql/actions/tools/src/",
  "ql/cpp/tools/src/",
  "ql/csharp/tools/src/",
  ...
]

When adding a new language, add its ql/{language}/tools/src/ entry to server/package.json files.

Exclude .qlx Files from npm

server/.npmignore must contain *.qlx to prevent compiled CodeQL query bytecode (which is OS/architecture-specific) from being included in the npm package.

Server Logger Writes to stderr Only

All logger.info/warn/error/debug methods write to stderr via console.error. This is required because in stdio transport mode, stdout is reserved exclusively for the MCP JSON-RPC protocol. Any non-protocol bytes on stdout corrupt the message stream.

CODEQL_PATH Environment Variable

The server resolves the CodeQL CLI binary at startup via resolveCodeQLBinary() in cli-executor.ts. The CODEQL_PATH env var takes an absolute path to the codeql binary, bypassing PATH lookup. This is critical for users who have multiple CodeQL CLI versions installed.

Publishing: codeql pack publish

• Use --threads=-1 (leave 1 core unused) for parallel compilation
• GITHUB_TOKEN env var is recognized automatically — no need for --github-auth-stdin
• Precompilation is enabled by default (only --no-precompile opt-out exists)
• The codeql pack install subcommand does not have a --threads flag

LICENSE File Name

The actual license file is LICENSE (no .md extension). Workflow steps and documentation must reference LICENSE, not LICENSE.md.

Helper Script

See verify-pack-compatibility.sh for automated compatibility checking.

Related Skills

• add-mcp-support-for-new-language - Adding support for new CodeQL languages
• validate-ql-mcp-server-tools-queries - Validating tool query functionality
• maintain-changelog - Updating CHANGELOG.md when preparing a release