Spec Upgrade

Upgrading the spec and regenerating the client

Nothing tells this repo when the backend's spec moves. Someone notices, and then runs this pipeline. The steps are ordered because each one's output is the next one's input, and because two of them are the only thing standing between a stale copy and a client that looks generated but isn't.

The pieces

Upstream, the spec is produced by the backend that serves these endpoints (Zipstack/unstract, manage.py generate_docstudio_spec, committed at specs/docstudio-oss.json). It is a copy here, not a fork.

The sequence

1. Copy the spec byte-for-byte from the backend commit you are upgrading to. Do not reformat it and do not edit it here — the drift gate regenerates from whatever is committed, so an edit here becomes a client that no backend serves.

2. Move SPEC_SOURCE in tools/gen_sdk.sh in the same commit. It records the backend repo, path, revision and sha256. Without that update, nothing distinguishes a current copy from one the backend has moved past — the script and the drift gate both report clean either way, which is exactly the failure this pipeline exists to prevent.

3. Regenerate:

   ./tools/gen_sdk.sh
   

   It rebuilds the tree from scratch with a pinned generator. If it exits non-zero on a generator warning, believe it: the generator downgrades a schema it cannot parse to a warning, drops the endpoint or model, writes the rest and exits 0 — so the warning is the only signal that an operation is missing.

4. Review the generated surface, with new files included:

   git add -N -- src/unstract/api_deployments/_sdk_docstudio
   git diff --stat -- src/unstract/api_deployments/_sdk_docstudio
   git diff -- src/unstract/api_deployments/_sdk_docstudio
   

   The --stat is the shape of the change; the full diff is the change. Read both — the operations and fields that moved only show up in the second.

   git add -N first because a plain diff cannot see a file the generator has newly created — which is precisely what a spec that grew an endpoint produces. Read the diff as the API change it represents: new modules are new operations, removed fields are removals.

5. Make the new surface reachable, in client.py. Regeneration only moves the transport. If the spec added an endpoint callers should be able to use, the facade is where it becomes public API. Fixes belong here or upstream in the spec, never in the generated tree — regeneration overwrites that wholesale.

   A new operation belongs to APIDeploymentsClient if it takes a deployment key, PlatformKeyClient if it takes a platform key, otherwise a new subclass of _HttpxFacade — never a free-standing class, or the transport, retry and exception translation get reimplemented and drift. The method is two lines:

   def list_widgets(self, org_id: str, *, page: int | None = None) -> dict[str, Any]:
       kwargs = list_widgets._get_kwargs(org_id, **{"page": page} if page else {})
       return self._read_or_raise(self._request_with_retry(**kwargs), "list_widgets")
   

   • Build from _get_kwargs, not sync_detailed: the generated _parse_response calls from_dict on an error body unguarded, so an undeclared one raises before the facade sees the status. Omit unset parameters rather than passing None — the builder renders some before it filters None out. Both are private to the generator, so pin them in tests/test_compat.py.
   • Send through _request_with_retry, and read through _read_or_raise, which checks the status first.
   • Return dict[str, Any]. The generated models are exported for callers who want typing; a hand-written TypedDict would not survive regeneration.
   • A new error type subclasses UnstractError.

   PlatformKeyClient.whoami is the smallest example in the tree.

6. Run the tests: uv run pytest tests/. tests/test_compat.py compares this client against the last released one, vendored under tests/baseline/. Refresh that baseline only when you mean to move the parity reference point, with tools/refresh_baseline.sh <released-version>; a spec upgrade on its own is not a reason to move it.

CI

sdk-drift regenerates from the committed spec and fails on any diff, so a hand-edit of the generated tree and a spec change nobody regenerated over both fail the same way. If it is red, run step 3 and commit the result.

Versioning and release

Choose the bump by what changed for callers: major when the spec removed or renamed something callers depend on, minor for new endpoints or new behaviour — a new facade method included — patch for fixes that keep the surface identical. A generated diff with removals in it is the signal for major — spec upgrades produce those. Behaviour the baseline pinned that has moved goes in ACCEPTED_DIVERGENCES in the same commit.

Do not touch __version__ in src/unstract/api_deployments/__init__.py in your PR. The in-repo value is the last released version; main.yml reads it, applies the bump you pick at dispatch time, and commits the result itself. Bumping it in the PR makes the release skip a version.

Release by dispatching Release Tag and Publish Package on main and choosing the bump. It publishes to PyPI before it tags — a failure after publish is retried by hand against a live artifact, not by re-publishing.

Downstream

unstract-cli pins this client exactly and vendors a copy of the same spec. After a release lands on PyPI, that pin needs bumping there; see the bump-client-pins skill in that repo.