Release Workflow
Guides the Backend.AI release process: version bump, changelog generation, and RC consolidation for final releases.
Parameters
- target_version (required): Release version (e.g.,
26.2.0, 26.2.0rc2)
- webui_version (optional): WebUI version to bundle
Workflow
Phase 1: Pre-flight
Parse parameters
- Extract
target_version from user input (required)
- Extract
webui_version if provided
Check current state
- Read
VERSION file for current version
- Verify clean working tree (
git status)
- Confirm on expected base branch (usually
main)
Determine release type
- Pre-release: version contains
rc, alpha, beta, or dev (e.g., 26.2.0rc1)
- Final release: clean semver (e.g.,
26.2.0)
Locate the changelog file
- The changelog is split per version branch:
CHANGELOG/{major}.{minor}.md
(26.9.0rc1, 26.9.0, and 26.9.1 all live in CHANGELOG/26.9.md)
- The root
CHANGELOG.md archives the releases made before the split — never write to it
- The file may not exist yet; the first release of the version branch creates it
For final releases, check for prior RC sections
- Scan
CHANGELOG/{major}.{minor}.md for sections matching ## {major}.{minor}.{patch}rc\d+
- List found RC sections to user
- These will be consolidated in Phase 3
For a {major}.{minor}.0rc1 target, settle the support level
- That form is the rc that cuts a new release line; the release script recognises it
on its own, so nothing else has to be classified by hand
- Ask the user whether the line is LTS (the March and September sprints are, per the
versioning scheme) — a year of support is a commitment and is never inferred
- If it is, pass
--lts to the release script below
Confirm with user before proceeding
- Show: target version, release type, WebUI version (if any), RC sections to consolidate (if any),
and for a branch cut, whether the line is LTS
- Wait for user approval
Phase 2: Release Script Execution
Run scripts/release.sh which performs these steps:
scripts/release.sh {target_version} [webui_version]
When the line being cut is LTS:
scripts/release.sh --lts {target_version} [webui_version]
To override the next development version (year rollover or planned sprint skip):
NEXT_DEV_VERSION={next_version} scripts/release.sh {target_version} [webui_version]
What the script does:
- Creates branch
release/{target_version}
- Downloads WebUI release (if
webui_version provided)
- Updates
VERSION file
- Freezes
NEXT_RELEASE_VERSION placeholders to the actual version (stable releases only; skipped for rc/a/b/dev/post)
- Runs
scripts/run-towncrier.py {target_version}, which consumes the changes/ fragments into CHANGELOG/{major}.{minor}.md (skipped entirely when there are no fragments)
- Runs
.github/scripts/update-maintained-versions.sh {target_version}, which registers the line when the target is an X.Y.0rc1 cut and retires the lines that are due — an LTS line once its recorded retire_after has passed, a regular line as soon as any newer line exists.
Report its output to the user: a retirement means that version stops receiving backports and drops out of the installer channel selection from that point on.
- Generates sample config files
- Generates API docs (OpenAPI, GraphQL schema)
- Runs quality checks:
pants tailor --check, pants check ::
- Commits everything as
release: {target_version}
- For sprint releases only (
{year}.{sprint}.0): advances NEXT_RELEASE_VERSION in meta.py to the next sprint and commits it separately as chore: bump NEXT_RELEASE_VERSION to {next_version}
NEXT_RELEASE_VERSION auto-advance (step 10):
- Runs only for sprint releases — patch must be
0 (e.g. 26.7.0). Patch releases (26.7.1) and pre-releases (rc/a/b/dev/post) are skipped automatically.
- Default: increments the sprint number, resets patch to
0 (26.7.0 → 26.8.0).
- Override the default by setting
NEXT_DEV_VERSION (e.g. 27.1.0 for a year rollover, or to skip sprints).
- Produces a second commit, kept separate from the
release: commit (freeze = this release; bump = next dev cycle).
Error handling:
- If quality checks fail, report errors and stop
- If towncrier fails, check
pyproject.toml towncrier config and changes/ directory
- If no changelog block was produced, check whether
changes/ held any fragment — run-towncrier.py skips towncrier when there is none, by design
- If config generation fails, check component CLI availability
- On any failure, suggest user fix the issue and re-run the script manually
Phase 3: Changelog Editing
For Pre-release (RC/alpha/beta/dev)
- towncrier generates a flat bullet list under
## {version} (date)
- No editing needed - the flat list is the final format for pre-releases
- Skip directly to Phase 4
For Final Release (RC Consolidation Required)
This is the core value of the skill. When releasing a final version (e.g., 26.2.0), all previous RC changelog sections (e.g., 26.2.0rc1, 26.2.0rc2) must be consolidated into the final version section.
Step 1: Collect RC entries
- Find all RC sections for the same major.minor.patch in
CHANGELOG/{major}.{minor}.md
- Pattern:
## {major}.{minor}.{patch}rc\d+ \(.*\)
- Extract all bullet items from each RC section, grouped by category (
### Features, ### Improvements, ### Fixes, etc.)
Step 2: Merge with new entries
- towncrier has already generated the new final version section with any remaining
changes/ fragments
- Combine: new entries + all RC entries, deduplicated by PR/issue number
- Group by category: Features, Improvements, Fixes, etc.
Step 3: Remove RC sections
- Delete the RC section blocks from
CHANGELOG/{major}.{minor}.md
- Only the final version section should remain for this patch level; sections for
earlier patch releases of the same version branch (e.g.
## 26.9.0 when
releasing 26.9.1) stay untouched below it
Step 4: Subsection grouping
- For each category (Features, Improvements, Fixes), group related items into
#### Subsection Title blocks
- Each subsection gets a 1-2 sentence description explaining the group
- Follow the established pattern from previous final releases (e.g., 26.1.0):
### Features
#### Fair Share Scheduler
Implemented a Fair Share scheduling system for equitable resource distribution.
* Add Fair Share row models with tests and migration ([#8008](https://...))
* Implement Fair Share repository layers ([#8030](https://...))
#### Other Features
* Standalone items that don't form a natural group ([#XXXX](https://...))
### Improvements
#### Storage Proxy Improvements
Introduced StorageTarget abstraction for flexible storage configuration.
* Introduce StorageTarget abstraction ([#7938](https://...))
#### Other Improvements
* Standalone improvement items ([#XXXX](https://...))
Grouping criteria (priority order):
Feature Initiative: Group PRs that serve the same goal or project, even if they span multiple layers (model, repository, service, API, CLI). A single initiative = a single group.
- e.g., "Fair Share Scheduler" — includes row models, repository, service, API, CLI, observer, sequencer
- e.g., "Sokovan Scheduler Redesign" — includes handler unification, status transitions, coordinator redesign
- e.g., "Email Notification System" — includes SMTP channel implementation + endpoint lifecycle notifications
System/Subsystem: Group PRs that improve a specific system across multiple areas under the system name.
- e.g., "RBAC System Improvements" — Creator/Purger/Granter, entity_fields, data migration
- e.g., "RBAC System Data Migration" — batch entity migration to RBAC DB
Domain Entity: Group PRs that target the same entity's Action/Repository/Service work under the entity name.
- e.g., "Scaling Group Management" — Create/Modify/Associate/Disassociate actions
- e.g., "Model Deployment Data Model Extension" — revision, autoscaling, policy row/repository/API
Technical Pattern: Group PRs that apply the same technical pattern across multiple entities under the pattern name.
- e.g., "Repository Pattern Standardization" — batch Admin*Repository consolidation
- e.g., "GraphQL DataLoader Extension" — DataLoader implementation for multiple entities
- e.g., "Code Quality and Consistency" — batch linter rule enablement, refactoring
Infrastructure: Group PRs that address infrastructure-level changes under the infrastructure area name.
- e.g., "Agent RPC Connection Pooling" — pool implementation + migration
- e.g., "CSV Export Infrastructure" — foundation implementation + per-domain exports
Grouping rules:
- If a PR could belong to multiple groups, place it in the most specific initiative group
- A group requires 2+ PRs, except for significant standalone features which may form a single-PR group
- Items that don't fit any group go under
#### Other {Category} (e.g., #### Other Features)
- Subsection descriptions should be past tense, summarizing the group's purpose/outcome in 1-2 sentences
- Never group by component (manager, agent, storage-proxy) — always group by functional area
Step 5: User review
- Present the reorganized changelog to the user
- Wait for approval or requested changes
- Iterate until user is satisfied
Step 6: Amend the release commit
git add CHANGELOG/{major}.{minor}.md
git commit --amend --no-edit
Phase 4: Summary
Report final status:
Release Prepared
Version: {target_version}
Branch: release/{target_version}
Type: {Final Release | Release Candidate | Pre-release}
WebUI: {webui_version or "not updated"}
Changelog: {N} entries across {M} categories
RC Merged: {list of merged RC versions, or "N/A"}
Error Handling
Release script failure
Release script failed at: {step}
Error:
{error_output}
Options:
1. Fix the issue and re-run: scripts/release.sh {target_version} [webui_version]
2. Run remaining steps manually
3. Abort: git checkout main && git branch -D release/{target_version}
No changes/ fragments
No news fragments found in changes/ directory.
This means no new changes since the last release.
If this is a final release consolidating RCs, this is expected - proceed to Phase 3.
Changelog merge conflict
Found duplicate entries during RC consolidation (same PR number in multiple RCs).
Deduplicated entries:
{list}
Using the latest version of each entry.
Examples
RC release
User: /release 26.2.0rc2
Agent: [Checks VERSION, runs release.sh, towncrier generates flat list, reports summary]
Final release with RC consolidation
User: /release 26.2.0
Agent: [Checks VERSION, finds rc1/rc2 sections in CHANGELOG/26.2.md, runs release.sh,
consolidates RC entries + new entries, groups into subsections,
presents for review, amends commit, reports summary]
Release with WebUI
User: /release 26.2.0 25.3.2
Agent: [Runs release.sh with WebUI version, full workflow]
Related
scripts/release.sh - The release automation script
scripts/run-towncrier.py - Runs towncrier against CHANGELOG/{major}.{minor}.md
scripts/changelog_files.py - Version to changelog path mapping
pyproject.toml - towncrier configuration (fragment types, template)
changes/ - News fragment directory
CHANGELOG/ - Per-version-branch changelogs; root CHANGELOG.md archives the pre-split releases
/submit - For regular PR submissions (not releases)
1---2name: release3description: Guide the Backend.AI release process - run release.sh, generate changelog via towncrier, consolidate RC entries for final releases with subsection grouping.4---56# Release Workflow78Guides the Backend.AI release process: version bump, changelog generation, and RC consolidation for final releases.910## Parameters1112- **target_version** (required): Release version (e.g., `26.2.0`, `26.2.0rc2`)13- **webui_version** (optional): WebUI version to bundle1415## Workflow1617### Phase 1: Pre-flight18191. **Parse parameters**20 - Extract `target_version` from user input (required)21 - Extract `webui_version` if provided22232. **Check current state**24 - Read `VERSION` file for current version25 - Verify clean working tree (`git status`)26 - Confirm on expected base branch (usually `main`)27283. **Determine release type**29 - **Pre-release**: version contains `rc`, `alpha`, `beta`, or `dev` (e.g., `26.2.0rc1`)30 - **Final release**: clean semver (e.g., `26.2.0`)31324. **Locate the changelog file**33 - The changelog is split per version branch: `CHANGELOG/{major}.{minor}.md`34 (`26.9.0rc1`, `26.9.0`, and `26.9.1` all live in `CHANGELOG/26.9.md`)35 - The root `CHANGELOG.md` archives the releases made before the split — never write to it36 - The file may not exist yet; the first release of the version branch creates it37385. **For final releases, check for prior RC sections**39 - Scan `CHANGELOG/{major}.{minor}.md` for sections matching `## {major}.{minor}.{patch}rc\d+`40 - List found RC sections to user41 - These will be consolidated in Phase 342436. **For a `{major}.{minor}.0rc1` target, settle the support level**44 - That form is the rc that cuts a new release line; the release script recognises it45 on its own, so nothing else has to be classified by hand46 - Ask the user whether the line is LTS (the March and September sprints are, per the47 versioning scheme) — a year of support is a commitment and is never inferred48 - If it is, pass `--lts` to the release script below49507. **Confirm with user before proceeding**51 - Show: target version, release type, WebUI version (if any), RC sections to consolidate (if any),52 and for a branch cut, whether the line is LTS53 - Wait for user approval5455### Phase 2: Release Script Execution5657Run `scripts/release.sh` which performs these steps:5859```bash60scripts/release.sh {target_version} [webui_version]61```6263When the line being cut is LTS:6465```bash66scripts/release.sh --lts {target_version} [webui_version]67```6869To override the next development version (year rollover or planned sprint skip):7071```bash72NEXT_DEV_VERSION={next_version} scripts/release.sh {target_version} [webui_version]73```7475**What the script does:**761. Creates branch `release/{target_version}`772. Downloads WebUI release (if `webui_version` provided)783. Updates `VERSION` file794. Freezes `NEXT_RELEASE_VERSION` placeholders to the actual version (stable releases only; skipped for `rc`/`a`/`b`/`dev`/`post`)805. Runs `scripts/run-towncrier.py {target_version}`, which consumes the `changes/` fragments into `CHANGELOG/{major}.{minor}.md` (skipped entirely when there are no fragments)816. Runs `.github/scripts/update-maintained-versions.sh {target_version}`, which registers the line when the target is an `X.Y.0rc1` cut and retires the lines that are due — an LTS line once its recorded `retire_after` has passed, a regular line as soon as any newer line exists.82 **Report its output to the user**: a retirement means that version stops receiving backports and drops out of the installer channel selection from that point on.836. Generates sample config files847. Generates API docs (OpenAPI, GraphQL schema)858. Runs quality checks: `pants tailor --check`, `pants check ::`869. Commits everything as `release: {target_version}`8710. For sprint releases only (`{year}.{sprint}.0`): advances `NEXT_RELEASE_VERSION` in `meta.py` to the next sprint and commits it separately as `chore: bump NEXT_RELEASE_VERSION to {next_version}`8889**NEXT_RELEASE_VERSION auto-advance (step 10):**90- Runs only for sprint releases — patch must be `0` (e.g. `26.7.0`). Patch releases (`26.7.1`) and pre-releases (`rc`/`a`/`b`/`dev`/`post`) are skipped automatically.91- Default: increments the sprint number, resets patch to `0` (`26.7.0` → `26.8.0`).92- Override the default by setting `NEXT_DEV_VERSION` (e.g. `27.1.0` for a year rollover, or to skip sprints).93- Produces a second commit, kept separate from the `release:` commit (freeze = this release; bump = next dev cycle).9495**Error handling:**96- If quality checks fail, report errors and stop97- If towncrier fails, check `pyproject.toml` towncrier config and `changes/` directory98- If no changelog block was produced, check whether `changes/` held any fragment — `run-towncrier.py` skips towncrier when there is none, by design99- If config generation fails, check component CLI availability100- On any failure, suggest user fix the issue and re-run the script manually101102### Phase 3: Changelog Editing103104#### For Pre-release (RC/alpha/beta/dev)105106- towncrier generates a flat bullet list under `## {version} (date)`107- **No editing needed** - the flat list is the final format for pre-releases108- Skip directly to Phase 4109110#### For Final Release (RC Consolidation Required)111112This is the core value of the skill. When releasing a final version (e.g., `26.2.0`), all previous RC changelog sections (e.g., `26.2.0rc1`, `26.2.0rc2`) must be consolidated into the final version section.113114**Step 1: Collect RC entries**115- Find all RC sections for the same major.minor.patch in `CHANGELOG/{major}.{minor}.md`116 - Pattern: `## {major}.{minor}.{patch}rc\d+ \(.*\)`117- Extract all bullet items from each RC section, grouped by category (`### Features`, `### Improvements`, `### Fixes`, etc.)118119**Step 2: Merge with new entries**120- towncrier has already generated the new final version section with any remaining `changes/` fragments121- Combine: new entries + all RC entries, deduplicated by PR/issue number122- Group by category: Features, Improvements, Fixes, etc.123124**Step 3: Remove RC sections**125- Delete the RC section blocks from `CHANGELOG/{major}.{minor}.md`126- Only the final version section should remain for this patch level; sections for127 earlier patch releases of the same version branch (e.g. `## 26.9.0` when128 releasing `26.9.1`) stay untouched below it129130**Step 4: Subsection grouping**131- For each category (Features, Improvements, Fixes), group related items into `#### Subsection Title` blocks132- Each subsection gets a 1-2 sentence description explaining the group133- Follow the established pattern from previous final releases (e.g., 26.1.0):134135```markdown136### Features137138#### Fair Share Scheduler139Implemented a Fair Share scheduling system for equitable resource distribution.140141* Add Fair Share row models with tests and migration ([#8008](https://...))142* Implement Fair Share repository layers ([#8030](https://...))143144#### Other Features145* Standalone items that don't form a natural group ([#XXXX](https://...))146147### Improvements148149#### Storage Proxy Improvements150Introduced StorageTarget abstraction for flexible storage configuration.151152* Introduce StorageTarget abstraction ([#7938](https://...))153154#### Other Improvements155* Standalone improvement items ([#XXXX](https://...))156```157158**Grouping criteria (priority order):**1591601. **Feature Initiative**: Group PRs that serve the same goal or project, even if they span multiple layers (model, repository, service, API, CLI). A single initiative = a single group.161 - e.g., "Fair Share Scheduler" — includes row models, repository, service, API, CLI, observer, sequencer162 - e.g., "Sokovan Scheduler Redesign" — includes handler unification, status transitions, coordinator redesign163 - e.g., "Email Notification System" — includes SMTP channel implementation + endpoint lifecycle notifications1641652. **System/Subsystem**: Group PRs that improve a specific system across multiple areas under the system name.166 - e.g., "RBAC System Improvements" — Creator/Purger/Granter, entity_fields, data migration167 - e.g., "RBAC System Data Migration" — batch entity migration to RBAC DB1681693. **Domain Entity**: Group PRs that target the same entity's Action/Repository/Service work under the entity name.170 - e.g., "Scaling Group Management" — Create/Modify/Associate/Disassociate actions171 - e.g., "Model Deployment Data Model Extension" — revision, autoscaling, policy row/repository/API1721734. **Technical Pattern**: Group PRs that apply the same technical pattern across multiple entities under the pattern name.174 - e.g., "Repository Pattern Standardization" — batch Admin*Repository consolidation175 - e.g., "GraphQL DataLoader Extension" — DataLoader implementation for multiple entities176 - e.g., "Code Quality and Consistency" — batch linter rule enablement, refactoring1771785. **Infrastructure**: Group PRs that address infrastructure-level changes under the infrastructure area name.179 - e.g., "Agent RPC Connection Pooling" — pool implementation + migration180 - e.g., "CSV Export Infrastructure" — foundation implementation + per-domain exports181182**Grouping rules:**183- If a PR could belong to multiple groups, place it in the most specific initiative group184- A group requires 2+ PRs, except for significant standalone features which may form a single-PR group185- Items that don't fit any group go under `#### Other {Category}` (e.g., `#### Other Features`)186- Subsection descriptions should be past tense, summarizing the group's purpose/outcome in 1-2 sentences187- Never group by component (manager, agent, storage-proxy) — always group by functional area188189**Step 5: User review**190- Present the reorganized changelog to the user191- Wait for approval or requested changes192- Iterate until user is satisfied193194**Step 6: Amend the release commit**195```bash196git add CHANGELOG/{major}.{minor}.md197git commit --amend --no-edit198```199200### Phase 4: Summary201202Report final status:203204```205Release Prepared206207 Version: {target_version}208 Branch: release/{target_version}209 Type: {Final Release | Release Candidate | Pre-release}210 WebUI: {webui_version or "not updated"}211 Changelog: {N} entries across {M} categories212 RC Merged: {list of merged RC versions, or "N/A"}213```214215## Error Handling216217### Release script failure218```219Release script failed at: {step}220221Error:222{error_output}223224Options:2251. Fix the issue and re-run: scripts/release.sh {target_version} [webui_version]2262. Run remaining steps manually2273. Abort: git checkout main && git branch -D release/{target_version}228```229230### No changes/ fragments231```232No news fragments found in changes/ directory.233234This means no new changes since the last release.235If this is a final release consolidating RCs, this is expected - proceed to Phase 3.236```237238### Changelog merge conflict239```240Found duplicate entries during RC consolidation (same PR number in multiple RCs).241242Deduplicated entries:243{list}244245Using the latest version of each entry.246```247248## Examples249250### RC release251```252User: /release 26.2.0rc2253Agent: [Checks VERSION, runs release.sh, towncrier generates flat list, reports summary]254```255256### Final release with RC consolidation257```258User: /release 26.2.0259Agent: [Checks VERSION, finds rc1/rc2 sections in CHANGELOG/26.2.md, runs release.sh,260 consolidates RC entries + new entries, groups into subsections,261 presents for review, amends commit, reports summary]262```263264### Release with WebUI265```266User: /release 26.2.0 25.3.2267Agent: [Runs release.sh with WebUI version, full workflow]268```269270## Related271272- `scripts/release.sh` - The release automation script273- `scripts/run-towncrier.py` - Runs towncrier against `CHANGELOG/{major}.{minor}.md`274- `scripts/changelog_files.py` - Version to changelog path mapping275- `pyproject.toml` - towncrier configuration (fragment types, template)276- `changes/` - News fragment directory277- `CHANGELOG/` - Per-version-branch changelogs; root `CHANGELOG.md` archives the pre-split releases278- `/submit` - For regular PR submissions (not releases)