# Prepare Release

> Prepare a new release of inspect_evals by creating a release branch, collecting changelog fragments, and opening a PR. Use when user asks to cut/prepare/create a new release or version bump.

- Skill: `ukgovernmentbeis/prepare-release` (Agent Skill)
- Install (CLI): `npx skillmds@latest add ukgovernmentbeis/prepare-release`
- Raw SKILL.md: https://api.skillmd.com/api/skills/ukgovernmentbeis/prepare-release/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: UKGovernmentBEIS (https://skillmd.com/u/ukgovernmentbeis)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/ukgovernmentbeis/prepare-release

---


# Prepare a Release

This workflow prepares a new release of `inspect_evals`. It creates a release branch, collects changelog fragments, bumps the version tag, and opens a PR. After merge it tags the merge commit and creates a GitHub release.

Reference: `PACKAGE_VERSIONING.md`

## Prerequisites

- The `gh` CLI must be authenticated
- You must have push access to the repository
- There must be changelog fragments in `changelog.d/` (beyond `.gitkeep` and `TEMPLATE.md`)

## Phase 1 — Prepare the release branch

1. **Ensure `main` is up to date**:

   ```bash
   git fetch origin
   git checkout main
   git merge --ff-only origin/main
   ```

   If the merge fails, stop and inform the user that their local `main` has diverged from `origin/main`.

2. **Check for changelog fragments**:

   List files in `changelog.d/` excluding `.gitkeep`, `TEMPLATE.md`, and `README.*`. If there are no fragments, stop and tell the user there is nothing to release.

3. **Determine the new version**:

   1. Find the current version from the latest `v*` git tag:

      ```bash
      git tag --sort=-v:refname --list 'v*' | head -1
      ```

   2. Ask the user what kind of bump this is, presenting the semver table from `PACKAGE_VERSIONING.md`:

      | Component | When to bump     | Examples                                                    |
      | --------- | ---------------- | ----------------------------------------------------------- |
      | **Major** | Breaking changes | Removing an eval, API changes, scorer output format changes |
      | **Minor** | New features     | Adding new evals, new task parameters, new utilities        |
      | **Patch** | Bug fixes        | Eval fixes, scorer fixes, dataset loading fixes             |

   3. Compute the new version string (e.g. `0.4.0`, `0.3.107`). Confirm with the user.

4. **Create the release branch**:

   The branch name is `release-YYYY-MM-DD` using today's date.

   ```bash
   git checkout -b release-YYYY-MM-DD
   ```

## Phase 2 — Collect the changelog

1. **Run scriv collect**:

   ```bash
   uv run scriv collect --version <NEW_VERSION>
   ```

   This removes the individual fragment files from `changelog.d/` and prepends a new section to `CHANGELOG.md`.

2. **Present the changelog diff to the user for review**:

   ```bash
   git diff CHANGELOG.md
   ```

   Tell the user: *"Please review the collected changelog above. You can edit `CHANGELOG.md` directly — let me know when you are satisfied, or tell me what changes to make."*

   **Do not proceed until the user confirms the changelog is ready.**

3. **Normalise the collected changelog**:

   `changelog.d/` fragments are excluded from the `mdformat` and `markdownlint-fix` pre-commit hooks (fragments are partial documents), so a hand-edit during review — or an unusual fragment — can leave `CHANGELOG.md` in a state those hooks would rewrite. That turns the Markdown Lint and README check jobs red on `main` after the release PR merges (see #2225, #2329). Run the hooks over `CHANGELOG.md`, then confirm a second pass is clean:

   ```bash
   uv run --group dev pre-commit run mdformat --files CHANGELOG.md || true
   uv run --group dev pre-commit run markdownlint-fix --files CHANGELOG.md || true
   # A second pass must be clean: applying fixes is fine, failing to
   # converge means a real error that would still break main.
   uv run --group dev pre-commit run mdformat --files CHANGELOG.md
   uv run --group dev pre-commit run markdownlint-fix --files CHANGELOG.md
   ```

   If the second pass still reports changes, resolve them before committing.

4. **Stage and commit**:

   ```bash
   git add CHANGELOG.md changelog.d/
   git commit -m "Prepare release v<NEW_VERSION>"
   ```

## Phase 3 — Push and open a PR

1. **Push the branch**:

   ```bash
   git push -u origin release-YYYY-MM-DD
   ```

2. **Open a draft PR**:

   <!-- markdownlint-disable-next-line no-space-in-code -->

   Extract the new version's section from `CHANGELOG.md` (everything from the version heading up to but not including the next `##` heading). Use this as the PR body:

   ```bash
   gh pr create --draft \
     --title "Release v<NEW_VERSION>" \
     --body "<EXTRACTED_CHANGELOG_SECTION>"
   ```

   **Important**: The PR title **must** start with `Release v` (e.g. `Release v0.4.0`) — this is how the `release-on-merge.yml` workflow identifies release PRs.

   Tell the user the PR URL and that it is in draft. They should mark it ready for review when appropriate.

## Phase 4 — After the PR is merged (automated)

Once the release PR is merged into `main`, the `.github/workflows/release-on-merge.yml` GitHub Actions workflow automatically:

1. Tags the merge commit with `v<NEW_VERSION>`
2. Creates a GitHub release with the changelog section as the body
3. Builds and publishes the package to PyPI
4. Notifies Slack

No manual action is required. The package version is derived from the new tag by `setuptools_scm`.

## Notes

- The package version is derived entirely from git tags via `setuptools_scm` — there is no version file to edit.
- If something goes wrong mid-workflow, the release branch can be deleted and the process restarted.
- The `v` prefix on tags is required (e.g. `v0.3.107`, not `0.3.107`).
- The `weekly-release.yml` workflow can also automate Phases 1–3 on a schedule or via manual dispatch, creating the release branch and draft PR for you.