Rask Architecture

rask architecture (workspace planes + composition)

The single most rask-specific thing to get right: which plane a package lives in, and how entrypoints compose packages. Get this wrong and uv/bun resolution breaks (loudly for uv, silently for bun) or you smear a domain across layers. Defers Python idioms to writing-python, FastAPI routing to fastapi, container builds to dockerfile.

There is no projects/ layer. The Polylith-style projects/<name> composition stubs were removed (2026-07). Deployables are ordinary workspace members built by .docker/<name>.dockerfile running uv sync --frozen --package <name> against the root uv.lock. Never recreate per-deployable pyprojects or per-deployable locks.

When to use

• Adding, moving, or deleting a workspace member (packages/*, services/*, or anything under frontend/).
• Wiring a new HTTP service entrypoint or editing one.
• Editing any pyproject.toml members/dependencies, or frontend/package.json workspaces.
• "ModuleNotFoundError" / uv won't resolve a first-party import after you added code.
• Deciding where new code belongs: reusable lib, runnable component, or deployable.

The planes — one language per plane

The tree is split by language first, then by layer. Each globbed directory is single-language; that is what makes the globs safe (see the invariants).

Current Python packages: lineage-kit, ray-cluster-env (deps-only), ray-kit, service-kit, storage, validate. (ratch was DISSOLVED 2026-08-28 — open_ray-kernel.md; tracker was DELETED 2026-08-30 — a per-file transfer ledger that never found a consumer here, its job done by JetStream WORK_QUEUE retention plus Dapr Workflow's durable state; htr is not a package — it is runners/htr, sealed and outside every glob.) Current Python services (13): gateway, compute, controlplane, ingest, flows, notifications — plus the lance plane (catalog, lineage, medallion, maintenance, viewer, search, annotator). (notifications is the per-subject inbox behind the bell, :8850, app-id notifications — see rask-services-fleet.) (compaction was renamed maintenance in 06cc7579 — it compacts, optimizes indices, cleans up old versions and reconciles cross-store drift, so it is named for all four, not one. core/core_api/search_api/volumes_api died in the R6/R20 media wave.) Current sealed runners (9): asr, assist, diarize, dummy, htr, insid3, kg, topics, voiceprint. (dummy is the GPU-free lane prover — real CDF read → merge_insert → fragment commit, no model download; insid3 is pinned to python 3.10.) Current JS packages: api, config, dockview, engine, explorer-api, flow, labeling, ui, zone-contract — see rask-frontend. Current zones (7, verified against frontend/microfrontends/ 2026-08-09): home (catch-all, base ''), annotator, compute, lakehouse, explorer, studio, models — each based at a bare /<zone>. models REPLACED train (on train's port 5178); a leftover microfrontends/train/ on a dev host is untracked build residue, not a zone. overview/discover/storage are retired; /storage and /data are routes inside lakehouse. Current deployables (each = a workspace member + a .docker/<name>.dockerfile): gateway, compute (R22 — compute on every surface), controlplane, ingest, notifications, runner, assist-runner — plus the one parametrized frontend.dockerfile built per zone (images tagged web-<zone>:<tag>), ray-cluster.dockerfile (the Ray head/Serve image) and rest-catalog.dockerfile, which is ONE image for seven lance services (catalog, lineage, medallion ×2 apps, maintenance, viewer, search, annotator) run with different commands. NB make k3s-build's COMPOSE_IMAGES is still only gateway compute controlplane — ingest has a dockerfile and a chart Deployment (:8830) but is not in that build loop yet.

The composition seam: three app factories, one per plane

service_kit.make_service_app(*, title, routers, proxy_router=None, lifespan=None) builds the FastAPI app with shared config/handlers/middleware. The lifespan is injectable: stateless services get the minimal default_lifespan (settings only); stateful ones pass a LifespanFactory (Lance/Ray/S3). Routers mount under settings.api_prefix; proxy_router mounts at root.

⚠️ THREE FACTORIES ARE SANCTIONED — know which plane you are in. This section used to say two layouts, and that the seven non-fleet services "construct FastAPI(...) directly with bespoke lifespans"; that was true and is not any more. Twelve of the thirteen services now come out of a factory, and which one is decided by the plane, not by taste:

gateway is in none of them — it builds FastAPI(...) itself, because it is a proxy, not a router host. The media three also share ONE lifespan (service_kit.media.lifespan.make_media_lifespan), with setup/teardown hooks for what is genuinely per-service (the annotator's actor plane).

The non-fleet services keep the fastapi skill's api/v1/endpoints/ + core/ + services/ layout rather than the fleet's flat-module layout — that half of the old paragraph still holds. What changed is the ENTRYPOINT: a main.py that opens its own app = FastAPI(...) is now the exception to justify, not the norm (docs/DECISIONS.md "The Python estate audit" DUP-12 counted eight of them repeating one boot, and the copies had drifted — the medallion stage runner had lost its request-id layer entirely).

A thin fleet-layout entrypoint is ~20 lines — import routers + a lifespan from the domain package, call the factory. compute/__init__.py:

from compute import health, proxy, routes
from compute.lifespan import make_lifespan
from service_kit import make_service_app

app = make_service_app(
    title="compute",
    routers=[health.router, routes.router],
    proxy_router=proxy.router,
    lifespan=make_lifespan,
)

The core husk is GONE (R6/R20, 2026-07-28)

services/core + services/core_api (the post-P7a transitional husk) are deleted, with search_api and volumes_api. Their capabilities live in the explorer plane: the S3 object browser is the viewer's objects.py endpoints (/api/explorer/object*); lines/EAD FTS re-land as catalog-governed Lance tables behind /api/explorer/search (docs/architecture/lance-ns-merge.md R6). Do not resurrect them.

Hard invariants (the gotchas)

• Workspace membership is globbed — and that is only safe because every globbed dir is single-language. Root pyproject.toml has [tool.uv.workspace] members = ["packages/*", "services/*"]; frontend/package.json has workspaces = ["microfrontends/*", "packages/*"] (paths relative to frontend/). Drop a directory in the right plane and it is a member — no manifest edit. The safety condition is the language purity, and the two toolchains fail asymmetrically when it breaks: a dir under a uv glob without a pyproject.toml is a hard error (Workspace member … is missing a pyproject.toml), fixable only by an exclude list (enumeration by another name); a dir under a bun glob without a package.json is SILENTLY skipped — bun prints "Done!" and the package is simply never installed, built, linted or tested, and nothing says so. So: never put a JS package under root packages//services/, and never put a Python package under frontend/. (The root manifest also notes runners/* is deliberately matched by no glob — sealed model envs whose heavy pins must never enter the fleet's resolution.) See references/adding-a-package.md.
• One lock. The root uv.lock is the only Python lockfile — dev, tests, and every fleet docker image resolve from it (uv sync --frozen --package <name>). The sealed runners/htr project carries its own lock and is invoked via uv run --project runners/htr runner (in-cluster the ray image ships the console script on PATH).
• service-kit keeps a light base. Base deps are storage, fastapi, pydantic, pydantic-settings, python-dotenv, dapr>=1.18.1, and 8 OpenTelemetry packages — the SDK, the OTLP/HTTP exporter, and instrumentors for fastapi, httpx, logging, requests, grpc and aiohttp-client. The last three landed 2026-08-23: the fleet runs bare uvicorn with no opentelemetry-instrument launcher, so whatever setup_otel names is ALL the instrumentation it gets, and without grpc + aiohttp the app→sidecar hop carried no traceparent and every Dapr span rooted a new trace. The heavy Lance/Ray deps live behind the [governed] / [lakehouse] / [lancekit] extras — keep them there. Never add lancedb, ray, or sqlmodel to the base: service-kit is shared by every service including the storeless ones (gateway via setup_otel, compute).
• known-first-party is COMPLETE — keep it that way. This bullet used to read "stale and silently drifting": the list held 9 of the 19 real first-party import names, so ten modules sorted into the THIRD-PARTY block. Closed 2026-08-30 (docs/DECISIONS.md "The Python estate audit" X3) as one pass — every name added, then uvx ruff check --select I --fix, which re-sorted 391 files. All 19 are listed now (6 code-shipping packages + 13 services). Step 4 of references/adding-a-package.md is the step that was being skipped on every landing: add the import name in the SAME change that adds the member. One name at a time is cheap; letting ten accumulate is a repo-wide re-sort again.
• Membership is globbed; TEST ENROLMENT IS NOT — and the asymmetry is where suites go missing. A directory dropped into packages//services/ is a workspace member with no manifest edit, but [tool.pytest.ini_options] testpaths is an EXPLICIT list. So a new member's tests/ runs nowhere until someone adds the path, and the run stays green while it does. Three suites landed green-by-absence this way (services/catalog, services/lineage — one pinning a privilege escalation, one a commit duplication — enrolled 2026-08-09). tests/unit/test_invariants.py::test_every_workspace_test_directory_is_in_the_root_testpaths now gates it in both directions, but ONLY over packages/*/tests and services/*/tests: a new top-level tests/<x>/ is still ungated, which is exactly how tests/e2e-py was lost once. Measured 2026-08-22: services/search and services/viewer shipped no tests at all (packages/ratch was the third — dissolved 2026-08-28); both have since gained suites (test_search_is_governed, test_the_search_door_is_wired, the viewer's gating suites), and the residue is tracked in the lakehouse register, row Q3-37 (drained 2026-09-10; in git history)/Q3-38 (the blanket ruff exemption, and ray_kit.submit untested), not here.
• A sealed runner's tests are invisible to the root pytest, and to CI. runners/* is matched by no glob by design, so make test names runners/htr and make test-slow names htr + dummy — by hand. dagger call test runs the root testpaths only and says so in its own doc comment, so the 75 test functions that exist in the runners execute in no CI job. Seven of the nine ship no tests at all. A lockfile's absence in those seven is NOT a defect — see the plane table above: a runner carries a uv.lock only where it builds an image.
• Do not resurrect viewer or control. The monolithic viewer service was dissolved (2026-06) into the gateway + per-domain services. There is no control package. New domain code lands in an existing package or a new one — never a revived monolith.
• viewer now means the lance media viewer (services/viewer, :8101) — the old rask viewer monolith and the RASK_VIEWER_* settings are gone.

When to load each reference

Sibling skills

rask-services-fleet (the gateway + per-service routing) · rask-frontend (zones, data, gates) · rask-styling (@rask/ui) · rask-lance-catalog (the catalog, governance, maintenance).

A runner's internals are deliberately undocumented here: each runners/<workload> is sealed and owns its own pipeline, models and GPU packing. There is no per-workload skill — one would make that modality look privileged, which is the opposite of how this platform is built.