all publishers

probabl-ai

@probabl-ai source repo

40 published skills

  1. ▌
    Export Ml Project · probabl-ai
    Coordinate executed notebooks and an offline MkDocs documentation site. Trigger on a generic "export" request, or to flip the notebooks/site gates after setup.
    145 repo stars
  2. ▌
    Manage Ml Backlog · probabl-ai
    Record experiment outcomes and triage idea files. A promoted idea moves from the Ideas table into a Backlog row. Record an experiment outcome in History and triage idea files. Summarize open, discarded, and aside ideas in the JOURNAL Ideas table. Move a row into Backlog when it is promoted, and keep the idea file. Also supports the model-entry selection mode: show real B<N> rows supplied by the deterministic CLI and consume one into a proposal. Trigger after audit, when a run finishes, when the user asks what to try next or to triage idea files, or when model-ml-pipeline routes its Backlog choice here. This is cadence, not a methodology owner.
    145 repo stars
  3. ▌
    Model Ml Pipeline · probabl-ai
    Require locked problem framing, write the first baseline design note for approval, then coordinate later choices, build, smoke testing, evaluation, and audit. On a first landing, write the locked baseline design note and ask for approval. Later landings offer only the choices the workspace justifies (an EDA proposal, Backlog, discussion). A missing modeling lock loads `frame-ml-problem` and stops. After a design is approved, coordinate build (pytest smoke is a build sub-step), the user's Evaluate (Recommended) / Modify / Stop gate, evaluation, and audit. Not for a single action already owned by evaluate, audit, or smoke debugging.
    145 repo stars
  4. ▌
    Review Ml Choices · probabl-ai
    Show stored project choices and re-enter the skill that can change one. Trigger when the user asks what we decided, what the current settings are, or to change a stored project choice. SKIP an explicit sync, export, or “a constraint changed” request — those load sync-ml-reports, export-ml-project, or frame-ml-problem directly. HOW TO USE: run `review choices`, show that board, then load one owning skill. Do not policy set from here.
    145 repo stars
  5. ▌
    Add Python Package · probabl-ai bundle
    Add a dependency through the project env manager, or ask the user when the environment is user-managed. Trigger when a workflow needs a missing import, after choose-python-library picks a library, or for editable install of src/<pkg>/. SKIP choosing between competing libraries (choose-python-library owns that). SKIP bootstrapping a missing manager directly (setup-python-env). When env.managed is null, load setup-ml-project if it is installed. HOW TO USE: read status.policy.env.managed, env detect, and env route. Classify this turn first (editable | add-skore | named package). If managed is null, load setup-ml-project and stop. If managed, env add --execute (or --editable). If unmanaged, ask; default is the user handles it — do not wait. When setup-ml-project already chose user-managed this turn, name the package and the manager, then return. Do not ask and do not wait. Never --execute while managed is false.
    145 repo stars
  6. ▌
    Export Ml Notebook · probabl-ai
    Write a jupytext percent file out as a notebook with empty outputs. `data_analysis/`, `experiments/`, and `audit/` are filled with empty outputs. Pass `--html` when that notebook should also become a viewer on the site. Kernel `notebook convert` stays for a source that is itself the execution. Trigger on notebook, ipynb, HTML, or executed-report requests, or when export-ml-project dispatches notebooks.
    145 repo stars
  7. ▌
    Evaluate Ml Pipeline 2 · probabl-ai bundle
    Evaluate one sklearn-compatible learner with the locked validation scheme and persist structured skore reports. The splitter and any non-default score are already on the DataOp from `build-ml-pipeline`. This skill writes the call without `splitter=`, persists the report, and records the locator. When the locked split is the training table and test table that shipped with the data, fit on the training table, then pass `splitter="prefit"` with only the test table. When `model-ml-pipeline` dispatched this turn, return the locator. A standalone turn owns the evaluate-stage close. TRIGGER when code calls `cross_val_score`, `cross_validate`, `classification_report`, `.skb.cross_validate`, or a handwritten metric print, or the user asks to score, evaluate, or run CV on one learner. Narrative reads of a persisted report belong to `audit-ml-pipeline`. HOW TO USE: before any evaluation call. Resolve G-SKORE-MODE, read the stops, and emit Pre-flight before code. Confirm symbols with `python -m skore_skills api get`.
    145 repo stars
  8. ▌
    Research Ml Practice · probabl-ai bundle
    Literature research for an ML methodology concern. Trigger when explore-ml-data or build-ml-pipeline load this skill. Not for routine profiling or a single API signature — use `api get` for symbols. Not a session owner — callers distill and write project files. HOW TO USE: skip intake fields the caller already supplied. Abstract the problem class before searching (never the dataset proper name). Survey extra analyses or depth a named concern; search distinct angles, fetch primary sources, follow up if thin, write `scratch/research/`. Return the path plus one or two sentences. Never mutate raw data, pick the final learner, or write `data_analysis.md` / the design note. Chat stays path + those sentences even when a harness asks for a complete answer in the message.
    145 repo stars
  9. ▌
    Review Ml Experiment · probabl-ai
    Read the stored report, then write one idea file and one Ideas row per candidate. Trigger after a successful evaluate, on "review this stem", or when review consent is audit or proceed. Do not write History, Backlog, or a design note.
    145 repo stars
  10. ▌
    Search Ml Literature · probabl-ai
    Search scientific and technical sources and write one idea file and one Ideas row for the direction the user confirms. Comes after review, beside the backlog. Do not call research-ml-practice. Do not write History, Backlog, or a design note. Do not pick a winner.
    145 repo stars
  11. ▌
    Choose Python Library · probabl-ai
    Resolve a library choice and add the selected dependency. Trigger for competing-library questions or an optional package not already fixed by the project stack — at first use (tabular exploration, pipeline, or eval extras), never during workspace setup.
    145 repo stars
  12. ▌
    Smoke Test Ml Pipeline 2 · probabl-ai
    Structural pytest: prediction count must match the predict-grid row count. TRIGGER when an approved experiment needs its smoke test, `pytest tests/smoke/` fails on row count, the user asks why the smoke test is failing, a pipeline edit needs proof, or the experiment script changes pipeline shape. STOP when `status.setup.pending` is non-empty (load `setup-ml-project`), or when there is no approved design or no matching experiment script: explain and send the user to setup or stay in `build-ml-pipeline`. A declined git is not asked again. "Why is smoke failing?" stays here. This action does not cover regression tests or CV interpretation. Do not write `skore.evaluate`. HOW TO USE: `status`, read the journal and experiment, write `tests/smoke/test_NN_*.py`, then `python -m skore_skills smoke run --stem <stem>`. Red JSON means fix the pipeline in `build-ml-pipeline`. Do not loosen the assertion.
    145 repo stars
  13. ▌
    Setup Git · probabl-ai
    Initialize safe version control, including an optional commit of the tree before setup changes it. Trigger when the user asks to initialize version control, add ignore rules, or make the first commit. This action owns first-time git setup only. Before the first commit, ask once which unknown hidden files and review paths to keep tracked. When loaded for the pre-change snapshot, commit the current tree before other setup writes, then return.
    145 repo stars
  14. ▌
    Export Ml Site · probabl-ai
    Package workspace markdown and existing notebook HTML into an offline MkDocs site. Embeds existing notebook HTML companions in their associated reports. Never executes Python. Trigger when the user asks for a website, mkdocs, or documentation site.
    145 repo stars
  15. ▌
    Persist Ml Git · probabl-ai
    Commit the current loop stage when the end-turn hook says invoke. Trigger after `python -m skore_skills git end-turn --stage <stage>` returns action invoke, or when the user asks to commit this turn's work.
    145 repo stars
  16. ▌
    Plot Ml Figure · probabl-ai bundle
    Pick pandas, seaborn, plotly, or matplotlib before writing figure code. Trigger when explore-ml-data, evaluate-ml-pipeline, or audit-ml-pipeline is about to save or display a chart, or when the user asks for a plot. Not a session owner. Not for skore/sklearn Display plots that already exist on a report. HOW TO USE: first match on the tree (owned plot, interactive iframe HTML, seaborn statistical, pandas chart, else matplotlib). In a notebook: save PNG (or HTML) then leave the figure as the cell output; never `plt.close`. Convert polars at the plot boundary. Missing lib → add-python-package. Confirm symbols with `api get`. The caller names the figure directory.
    145 repo stars
  17. ▌
    Triage Ml Task · probabl-ai
    Session owner: list installed entry skills and ask which to run. Load a skill without asking only when the request is certain to be that skill. Trigger on an ambiguous request, a finished stage, a workspace-open session, or "what should we do next".
    145 repo stars
  18. ▌
    Explore Ml Data 2 · probabl-ai bundle
    Explore the dataset before designing any model. Place `data_analysis/data_analysis.py` and do not execute it. Run `materialize.py` once, write `data_analysis.md` and JOURNAL § Data understanding. Never design the model, edit `src/<pkg>/`, or modify raw data files. TRIGGER when the user asks to explore, profile, or understand the data; triage sent them here (`status.data_analysis` missing); a data source changed; they want to refresh a recorded EDA; or leakage or research arises on a recorded EDA. STOP when `status.setup.pending` is non-empty (load `setup-ml-project`), when there is no data (send to triage), the request is not raw-data exploration, or EDA is recorded with no refresh and no methodology concern. Do not invent a package root or a root `JOURNAL.md` while setup is pending. HOW TO USE: G-TABULAR via `add-python-package`, infer or ask the targets, load `plot-ml-figure` if installed, then ask the five-option continuation board, including Close.
    145 repo stars
  19. ▌
    Setup Workspace · probabl-ai bundle
    Detect or scaffold the standard ML workspace layout. Cookiecutter only: directories, README.md files, and src/<pkg>/ stubs. After a first scaffold, turn executed notebooks and the documentation site on by default (no ask). TRIGGER for a new ML project layout or first scaffold. SKIP pipeline, evaluation, exploratory data analysis, and library choice. HOW TO USE: detect first. For a fresh or manager-only layout, resolve G-PKG-NAME via `AskUserQuestion` unless `policy.package` or `src/<pkg>/` already names it, then scaffold, then persist notebooks/site true when both are still unset. For an existing layout, stop without scaffolding or inventing files.
    145 repo stars
  20. ▌
    Shape User Idea · probabl-ai
    Shape a user idea or a named artifact into one idea file and one Ideas row after they confirm. A topic with no artifact belongs to search-ml-literature. Do not write History, Backlog, or a design note. Do not pick a backlog row.
    145 repo stars
  21. ▌
    Sync Ml Reports · probabl-ai
    Copy skore reports between local, Hub, and MLflow, and optionally switch the upload destination. Trigger when the user asks to sync or migrate reports, switch skore mode, upload reports to Hub or MLflow, or pull Hub/MLflow reports onto disk. First G-SKORE-MODE pick stays evaluate-ml-pipeline.
    145 repo stars
  22. ▌
    Frame Ml Problem · probabl-ai bundle
    Record the problem, deployment setting, metric, baseline, and fold count before any model code. Ask every missing decision in one turn, from `frame show`. Does not write Python, estimator hyperparameters, or splitter constructors. TRIGGER when the user asks which metric to compare on, how new rows should be split, which baseline to use, or says a problem constraint changed. Not when they ask to run evaluation or CV. HOW TO USE: run `python -m skore_skills frame show`. Read each reference named in `questions` once, ask every key in `missing` in one message, and write every answered value. If a key is still unanswered, ask it and stop. If the write fills every required decision, set Status to `locked`, say those choices are reused and can be changed by name, then follow `proceed`. Do not ask to confirm. If that command is missing, or the recorded task is not classification, regression, or multi-output regression, follow Fallback below and do not invent the closed menu.
    145 repo stars
  23. ▌
    Setup Ml Project · probabl-ai
    Ask only the pending setup pieces. When Git is selected, commit the current files before setup writes only if that check finds files to preserve. Trigger when the user asks to set up or bootstrap a workspace, or when a pipeline stage finds status.setup.pending non-empty. The question tells the user to select each piece they want. It starts by saying that before any modeling, the workspace should be set up. Do not say the order the pieces will run. Do not offer a piece that is already done or declined. Then, in that same opening phase, ask the environment manager, whether we manage the env, the import name, and git autocommit — only for pieces the user selects, and answers that are not already recorded. Persist those answers. An empty folder or a clean tree records skipped and does not ask. Then run only what the user selects. Git may ask once, just before the first commit, which unknown files to keep. Return to the skill that dispatched this turn.
    145 repo stars
  24. ▌
    Setup Python Env · probabl-ai
    Detect the env manager and bootstrap the runtime, agent-tools, and composed development environments. Detect with `python -m skore_skills env detect`, persist the manager and `env.managed`, then `env init --manager`, `env sync --execute`, install Skore for a recorded hub or mlflow destination (plain Skore when unset or local), and `env verify --execute`. Does not add other stage ML libraries. TRIGGER when the user asks for the env manager, pixi, uv, or a Python environment, or when none is recorded yet. SKIP later packages (add-python-package). SKIP scaffolding src/ (setup-workspace). HOW TO USE: detect, ask managed vs user-managed, ask the manager when needed, then env init, env sync, add Skore, and env verify. When setup-ml-project dispatched this turn, skip each ask whose answer is already recorded. One recorded answer does not skip the other ask.
    145 repo stars
  25. ▌
    Audit Ml Pipeline 2 · probabl-ai bundle
    Audit a persisted skore report read-only and produce a reusable digest. Do not execute it. Run `materialize.py` once. Never call `skore.evaluate` or `project.put`. TRIGGER when a completed run needs an audit, the user asks to audit, show, or re-audit an experiment, a re-run overwrote the same `put()` key, or they want a narrative of a past experiment without `journal/ideas/` files. STOP when `status.setup.pending` is non-empty (load `setup-ml-project`), or when there is no approved design, no report, or no agent feature: explain and send the user to setup, `evaluate-ml-pipeline`, or `model-ml-pipeline`. Also stop for raw-data exploration or sourcing a future experiment. HOW TO USE: confirm journal, experiment, smoke, and report; place the file from `templates/audit.py`; copy `templates/materialize.py` to scratch; execute once; derive G-AUDIT-FINDING; then the continue-or-close gate. Resolve skore symbols with `api get`.
    145 repo stars
  26. ▌
    Build Ml Pipeline 2 · probabl-ai bundle
    Declare a skrub DataOps graph from the data source to the predictor after framing is locked. Stateless steps use `.skb.apply_func`. Stateful steps use `.skb.apply`. After the graph exists, attach the locked splitter and any non-default score. No fit, tune, persistence, or `skore.evaluate`. TRIGGER when writing or editing any link from data source to predictor (loaders, preprocessing, features, composition, the final estimator), including a pure-Python function on that path, a step added or reordered, a bare `sklearn.Pipeline` as the top-level, or a request to build a pipeline, classifier, or regressor. HOW TO USE: before the first declarative line and on every structural edit. Read the stops and emit Pre-flight before code. Confirm new names with `python -m skore_skills api get`.
    145 repo stars
  27. ▌
    Python API · probabl-ai bundle
    Look up the public API of a Python package against the *installed version* and cache what's worth keeping. Four shapes by question type: (0) cache hit under `scratch/api/<lib>/<version>/`; (1) `inspect.signature` + `pydoc.render_doc` for a symbol; (2) `dir` / `pkgutil.iter_modules` for a module surface; (3) WebSearch + WebFetch of versioned docs for narrative ("how", "which", "what does X return when Y"). Never write a symbol from training-data memory — recognition is not a lookup. TRIGGER — any of: - About to name a symbol (function / class / method / arg) in code. - User asks "what's the signature of X?", "what's in module Y?", "how do I call X?", "which of A/B should I use?". - User asks "what does X return when <condition>?" (Shape 3 — see decision table). - Another workflow skill (`build-ml-pipeline`, `evaluate-ml-pipeline`, `iterate-from-skore`, `smoke-test-ml-pipeline`) says "consult the API skill". - About to reach for a library's "obvious" pattern from memory. SKIP when: the signature is obvious from
    145 repo stars
  28. ▌
    Explore Ml Data · probabl-ai bundle
    Owns data understanding BEFORE any model is designed. Places and executes `data/eda.py` (a jupytext `# %%` script) via the shared in-process runner, reads the streamed digest, then writes a persisted `data/eda.md` report (plus linked `data/eda_<table>.html` skrub `TableReport` pages) and the `## Data understanding (EDA)` section of `journal/JOURNAL.md`. The point is to surface the dataset facts — shape, dtypes, missingness, cardinality, target balance / skew, datetime / group structure, feature associations — that JUSTIFY the later learner / splitter / metric decisions, so the user understands *why* the modelling choices are made. Uses `skrub.TableReport` for dataframe overviews and the shared runner `audit-ml-pipeline/scripts/run_cells.py`. Stops at "EDA executed, `data/eda.md` + HTML written, JOURNAL EDA section updated." Never designs the model, never edits `src/<pkg>/`, never modifies the user's raw data files. TRIGGER — any of: - `iterate-ml-experiment` § 0 bootstrap, BEFORE the baseline design note — th
    145 repo stars
  29. ▌
    Test Ml Pipeline · probabl-ai
    Owns the `tests/` folder of an ML workspace and the pairing rule between an experiment and its tests. Lightweight router: every test category has its own subskill (`smoke-test-ml-pipeline` is the only one for v1; `regression-test-ml-pipeline` / `distribution-test-ml-pipeline` etc. plug in as siblings as the workspace grows). This skill places an empty `tests/<category>/` folder, enforces the stem-pairing rule between `tests/<category>/test_NN_<short_name>.py` and `experiments/NN_<short_name>.py`, and dispatches to the matching subskill when the user asks for a test. TRIGGER when: a new design note was just approved by `iterate-ml-experiment` and the matching test has to be drafted before the experiment can be marked done; the user asks "write the smoke test for `02`", "add the regression test", "do we have a test for X?"; an experiment script was edited and the paired test needs revisiting; about to run `pytest tests/` and one of the expected paired tests is missing. SKIP when: the design note does not yet ex
    145 repo stars
  30. ▌
    Audit Ml Pipeline · probabl-ai bundle
    Owns the `audit/` folder: one `# %%` (jupytext percent) Python file per experiment, aligned 1:1 with `experiments/NN_<short_name>.py` and `journal/NN_<short_name>.md`, that loads the experiment's skore report **read-only** and uses bare-last-expression cells whose `__repr__` carries the audit's signal. The agent executes the audit file via the bundled in-process runner (`audit-ml-pipeline/scripts/run_cells.py` — IPython `InteractiveShell.run_cell`), which streams a markdown digest of each cell's stdout + last-expression repr to stdout (optionally also to a file). The digest fuels narrative work (the `JOURNAL.md` Status + History update, follow-up questions about a past experiment, cross-experiment comparison). Stops at "audit/NN_*.py is placed, executed, and the digest is available." Never calls `skore.evaluate(...)` or `project.put(...)`. TRIGGER — any of: - `iterate-ml-experiment` § 4 record-outcome — audit is dispatched FIRST (replaces scratch probes for metric extraction). - The user asks "audit experimen
    145 repo stars
  31. ▌
    Build Ml Pipeline · probabl-ai bundle
    Declare the pipeline from data source to predictor as a **skrub DataOps graph** (not as a bare `sklearn.Pipeline`). Every step is either a pure-Python function (stateless) attached via `.skb.apply_func`, or a sklearn-compatible estimator (stateful) attached via `.skb.apply`. Stops at the declared object — no fit, split, tuning, persistence, or evaluation. TRIGGER — any of: - Writing or editing code that declares any link in the chain *data source → predictor*: loaders, preprocessing, encoders / imputers / scalers, feature steps, composition objects (`Pipeline`, `ColumnTransformer`, skrub `tabular_pipeline`, `nn.Module`), or the final estimator. - A pure-Python data-processing function destined for the pipeline path (cleans / derives / reshapes) — whether wrapped via `FunctionTransformer`, `skrub.@deferred` / `skrub.var`, a custom `BaseEstimator` subclass, or just called in the training path before the estimator. - A step is added, removed, swapped, or reordered inside an existing pipeline declaration. - A bar
    145 repo stars
  32. ▌
    Iterate From User · probabl-ai
    Source the next ML experiment proposal from the user via one of three entry points selected by `AskUserQuestion`: (a) a scientific article URL the agent must read and synthesize, (b) a resource link or path (GitHub issue / spec file / reference repo), or (c) free-text the user types directly. In every branch, the agent reads the source, synthesizes its understanding of what to implement, and confirms with the user *before* returning the Proposal block. Hand the confirmed Proposal back to `iterate-ml-experiment`, which writes it into `journal/NN_short_name.md` and seeks the user's design-note approval. Stops at "Proposal returned, user-confirmed"; never writes a design note, never authors acceptance criteria. TRIGGER when: `iterate-ml-experiment` is picking a sourcing strategy and the user picks `user` from the menu; the user volunteers a concrete idea ("I want to try X"); the user pastes or links a scientific article, GitHub issue, spec file, or reference repo and asks us to read it. SKIP when: the user wants
    145 repo stars
  33. ▌
    Python Code Style · probabl-ai bundle
    Owns Python code style for this stack: ruff for lint + format, numpydoc for docstrings. Three responsibilities — (1) place the project's `ruff.toml` from the bundled template once the stack and workspace are in place, (2) run ruff against any Python files Claude has just generated or edited, and (3) contextualize each touched file's comments to the data-science problem — rewriting any leftover template / workflow prose (skill names, gates, runner, digest, guard-rails) into concise, problem-specific docs so the user's committed files read like a colleague wrote them, not like a generated scaffold. Stops at "the touched files pass `ruff check` and document the problem, not the process." TRIGGER when (any of these): (1) a Python file was just created or edited via Write / Edit / MultiEdit — invoke this skill before declaring the task done so ruff is run AND the file's comments are contextualized to the problem; (2) a fresh ML workspace was just scaffolded by `organize-ml-workspace` and the project has no `ruff.t
    145 repo stars
  34. ▌
    Iterate From Skore · probabl-ai
    Source the next ML experiment proposal by **reading the audit digest** at `scratch/audit/<stem>/audit.md` (produced by `audit-ml-pipeline` at § 4 record-outcome). For every row in the digest's `## Checks summary` whose `severity` is `issue` or `tip`, follow the row's `documentation_url` to draft a Backlog row whose `Item` is the mitigation the docs recommend. The `## Metrics summary` provides context for the human summary paragraph but does not drive Backlog rows on its own. Returns the enriched Backlog rows + a one-paragraph summary back to `iterate-ml-experiment`, which writes the rows into `JOURNAL.md` and re-presents the sourcing menu so the user can promote a `B<N>` row. Stops at "Backlog enriched, summary returned"; never writes a per-experiment design note, never picks the "winning" finding — the user picks via `B<N>`. TRIGGER when: `iterate-ml-experiment` is picking a sourcing strategy and the user picks `skore` from the menu; the user says "mine the report", "what does skore see?", "fill the backlog
    145 repo stars
  35. ▌
    Python Env Manager · probabl-ai bundle
    Single source of truth for "which Python environment manager does this project use, and how do I install a package with it?". Owns the detection table (pixi / uv / poetry / hatch / conda+mamba / pip+venv), the install / remove / upgrade commands per manager, and the bootstrap path when no manager is in place (default recommendation: pixi). Stops at "the install command was issued with the right manager and the package is importable". TRIGGER when (any of these): (1) **about to install / add / pin / upgrade / remove a Python package** — `pip install`, `pixi add`, `uv add`, `poetry add`, `conda install`, etc. — under any framing; (2) `data-science-python-stack` § "Missing dependency" surfaced a missing import and an install is the next step; (3) a workflow skill's Stop condition fired on a missing dependency (`build-ml-pipeline`, `evaluate-ml-pipeline`, `organize-ml-workspace`, `audit-ml-pipeline`); (4) starting a new Python project and no manager is in place yet (bootstrap with pixi unless the user picks other
    145 repo stars
  36. ▌
    Evaluate Ml Pipeline · probabl-ai bundle
    Methodology for evaluating a single sklearn-compatible learner (in particular, the `SkrubLearner` produced by `build-ml-pipeline`). Owns: which entry point to call (`skore.evaluate` first, the explicit report classes when needed), which cross-validator to pick from scikit-learn's catalogue, how to consume the structural metadata (`groups`, `times`, …) attached at build time via `.skb.mark_as_X(split_kwargs=...)`. Stops at "what does the report say". Defaults (metrics, plots) come from skore; only override on explicit user request. TRIGGER when: code calls `cross_val_score`, `cross_validate`, `classification_report`, or any handwritten metric print (`print(mean_squared_error(...))`); code calls `.skb.cross_validate(...)` (route through skore for richer output); user asks how to score, evaluate, or compare a single learner; user asks how to pick a cross-validator; user wants to see a report / metrics / diagnostic plots for a fitted learner. SKIP when: declaring the pipeline (use `build-ml-pipeline`); hyperparam
    145 repo stars
  37. ▌
    Iterate Ml Experiment · probabl-ai bundle
    Owns the iteration loop on top of an ML workspace: the `journal/JOURNAL.md` index and the per-experiment `journal/NN_short_name.md` design notes that must be drafted and approved by the user **before** `experiments/NN_short_name.py` is created. Drives the propose → iterate → approve → implement → record loop; dispatches to `iterate-from-skore` / `iterate-from-user` for sourcing. TRIGGER — any of: - A session opens in an ML workspace (whether or not `journal/` exists yet — missing/placeholder → bootstrap mode). - User says "what's next", "resume", "where were we", "let's iterate", "propose next", "first baseline". - About to create a new `experiments/NN_*.py` (the matching `journal/NN_*.md` must exist and be approved first). - User wants to record an outcome from a finished run. - User asks to compare past experiments or review what's been tried ("compare X and Y", "where are we?"). SKIP when: no `journal/` yet AND no workspace scaffold (route to `organize-ml-workspace`); the work is mechanical inside `pipelin
    145 repo stars
  38. ▌
    Organize Ml Workspace · probabl-ai bundle
    Decide where files live in an ML experimentation project: reusable code in `src/<pkg>/`, one `# %%` script per experiment in `experiments/`, design notes + index in `journal/`, reports in `reports/`, agent-only probes in `scratch/`. Owns the layout, the file-creation rules (one file per experiment, ask before editing), and the jupytext `# %%` script convention. Never imposes `data/` — the user owns that. TRIGGER — any of: - Starting a new ML project / scaffolding a workspace. - About to create the first experiment file in a project. - About to create `src/<pkg>/data.py` / `features.py` / `pipeline.py` / `evaluate.py` for the first time. - About to write a `.ipynb` for experimentation — redirect to a `# %%` script under `experiments/`. - User asks where something should live, how to organize the project, or how to set up the workspace. - About to add a new experiment iteration — decide new file vs edit existing (ask the user). SKIP when: the file is clearly part of an already-populated module (e.g., adding a f
    145 repo stars
  39. ▌
    Smoke Test Ml Pipeline · probabl-ai
    Owns the smoke test contract for an ML experiment: a small, diagnostic-by-construction pytest that fits the experiment's learner on a portion of the real `data/` source and predicts on a *disjoint* portion that deliberately carries **no pre-history buffer**. The assertion is structural — the number of predictions must equal the number of rows in the predict grid. A pipeline that loads-then-features-then-splits will silently drop the cold-start rows of the predict slice and the test will fail with a row-count mismatch; a pipeline that marks X early and references upstream history nodes from feature steps will pass trivially. The smoke test is the executable proof of the X-marker placement rule from `build-ml-pipeline`. TRIGGER when: `test-ml-pipeline` has dispatched here to write the smoke test for an approved experiment; `pytest tests/smoke/` is failing on row count; the user asks "why is the smoke test failing?"; a pipeline edit in `build-ml-pipeline` needs an executable proof; an experiment script changes t
    145 repo stars
  40. ▌
    Data Science Python Stack · probabl-ai bundle
    Opinionated Python stack for data-science / ML work — one library per job, organized into tiers (mandatory / user choice / optional / transitive). SKILL.md is the index; per-library `references/<library>.md` files carry scope, "pick this when" / "pick something else when", and pairings. TRIGGER when (any of these): (1) **a library import fails** in this stack's domain — the answer is install, not substitute (see § "Missing dependency"); (2) **a library choice has to be made** — explicitly (the user asks "which library for X?") or implicitly (code is about to introduce a new dependency, or the project is being scaffolded and the tabular library hasn't been picked yet); (3) starting a new Python data-science / ML project; (4) the user or current code reaches for a substitute outside the stack (xgboost, lightgbm, black, isort, flake8, poetry, hatch), or reaches for `mlflow` to log params/metrics, or for `cross_val_score` + handwritten reporting — redirect: tracking → `skore` Project API, evaluation / reporting →
    145 repo stars