Kelly Local Model Lab

Kelly Local Model Lab

Use Busabase as the durable data and control plane while a trusted local Worker performs compute on Apple Silicon. Fine-tuning changes behavior; it is not a replacement for RAG when the goal is to inject current knowledge.

Mandatory Dependencies

Before creating, provisioning, or changing the AirApp, read and follow:

1. $kelly-app-skill-creator for product behavior and UI quality;
2. $busabase for Space selection, node discovery, ChangeRequests, and trusted writes;
3. $busabase-app-creator for AirApp runtime, SDK, security, validation, and deployment;
4. $kelly-app-skill-creator-tests before claiming full conformance.

If a dependency is unavailable, keep the canonical local artifact and training pipeline intact, stop before the unavailable Busabase operation, and report the missing dependency. Never replace Busabase with SQLite, local JSON, or another persistent workflow backend.

Operating Boundary

• The AirApp reviews examples, runs, evaluations, and registered adapters. It never launches MLX, reads local model files, or performs training itself.
• A trusted local Agent/Worker claims one eligible training-runs record, downloads its immutable Drive snapshot, trains locally, writes heartbeats, uploads the adapter and report, and then marks the run terminal.
• Busabase Base records hold structured state and provenance. Drive/File holds dataset snapshots, adapters, reports, and model cards. Vault holds only named credential requirements; values never enter the browser.
• Base-model caches are disposable local compute caches. They are not canonical workflow state and need not be uploaded to Busabase.
• A deployed AirApp cannot call the Mac's localhost. Inference or training requests cross the same reviewed Busabase job boundary.

Read references/lab-schema.md before operating on resources, exporting data, claiming work, evaluating, or registering an adapter.

Default Experiment

For a first Apple Silicon validation, use:

• base model: mlx-community/Qwen3-0.6B-4bit for a fast smoke run;
• method: QLoRA through MLX-LM;
• task: short App-in-Skill brief to strict app_spec JSON;
• metrics: JSON validity, schema validity, exact-field accuracy, and latency;
• output: a LoRA adapter, never a rewritten foundation checkpoint.

Treat the bundled training/fixture/ data as a pipeline smoke fixture only. It never impersonates reviewed Busabase data. Production experiments must carry a Drive snapshot reference and SHA-256 hash on the run record.

Workflow

1. Inspect runtime readiness, the explicit Space, resource versions, and local Worker readiness. Do not read or mutate app resources before Space selection.
2. Review examples individually. Use approve, request_changes, or block; preserve the note, reviewer, timestamp, source, and content hash.
3. Export only approved examples. Split by source or another leakage-resistant grouping, write train.jsonl, valid.jsonl, and locked test.jsonl, upload one immutable snapshot to Drive, and record its SHA-256 digest.
4. Create a ready training run containing the exact base model and revision, method, snapshot reference/hash, and config. Never train from a mutable live query whose contents cannot be reproduced.
5. Claim the run atomically. Record claimant, attempt, claimed time, and heartbeat. Re-read eligibility before starting MLX.
6. Evaluate the untouched base model first, train the adapter, then evaluate the adapter on the identical locked test file and decoding settings.
7. Upload the adapter, full evaluation report, and model card. Record terminal status and sanitized errors. Do not mark a run done merely because MLX exited.
8. Let a human choose promote, hold, or reject on the adapter evaluation. Promotion creates or updates a model-registry candidate/active record with base revision, dataset hash, run id, and artifact reference.

Local Smoke Command

Use an isolated virtual environment. The installed MLX-LM version must be pinned in requirements-mlx.txt before a result is treated as reproducible.

python3 -m venv .venv
.venv/bin/python -m pip install -r requirements-mlx.txt
.venv/bin/python scripts/mlx_smoke.py

Outputs go under the ignored .cache/smoke-run/ directory. The script validates the fixture, hashes all splits, records the exact package version and command, evaluates the baseline, trains, evaluates the adapter, and writes report.json.

UI

• #/overview: readiness, approved split counts, active runs, candidate adapters, pipeline stages, latest baseline/adapter delta, and recent runs.
• #/examples[/<id>]: desktop list/detail review queue and separate phone list and detail panes with stable example ids and review actions.
• #/runs: base model, method, claim/status, heartbeat, artifacts, and failures.
• #/evaluations: locked baseline/adapter comparisons and promotion verdicts.
• #/registry: active, candidate, and retired adapters with complete lineage.
• #/settings: responsive Help & Settings modal with guide, resources, and sanitized training configuration.

Demo mode is explicit and in-memory only. ?demo=1 opens the overview; ?demo=dataset, ?demo=evaluations, and ?demo=registry open named screenshot scenes. lang=en or lang=zh forces the UI language. Demo never claims a real connection, writes Busabase, or proves the local Worker is online.

Completion

Do not claim a production fine-tune until:

• the snapshot came from reviewed Busabase examples and has a stable Drive ref/hash;
• the base model revision, MLX-LM version, command, and config are recorded;
• baseline and adapter used the same locked cases and decoding settings;
• adapter artifacts and reports exist and are referenced by a terminal run;
• the human promotion verdict is recorded against the evaluated version;
• app checks, local/OSS conformance, responsive browser checks, and available Cloud/AirApp suites passed or are explicitly reported as skipped.

Stop when the Space or dataset is ambiguous, examples lack review/provenance, the run changed after claim, test leakage is detected, the model license is incompatible, an artifact hash fails, or a secret would cross into the browser.