# Kaggle Research Compute

> Use when a research or engineering task needs automatic heavy-compute routing to free Kaggle Kernels through the local broker, with agent-driven push, poll, fetch, and a multi-run resume loop across concurrent kernels; free CPU (quota-free) and GPU under a self-imposed weekly GPU-hour cap.

- Skill: `hoanganhduc/kaggle-research-compute` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add hoanganhduc/kaggle-research-compute`
- Raw SKILL.md: https://api.skillmd.com/api/skills/hoanganhduc/kaggle-research-compute/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: hoanganhduc (https://skillmd.com/u/hoanganhduc)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/hoanganhduc/kaggle-research-compute

---


# Kaggle Research Compute


## Windows Runtime Commands

On native Windows, use the managed Windows runner and the native runtime command target. Set `$runtime` to the installed runtime root. Multi-agent installs usually use `%LOCALAPPDATA%\ai-agents-skills\runtime`. Then run:

```powershell
$runtime = if ($env:AAS_RUNTIME_ROOT) { $env:AAS_RUNTIME_ROOT } else { "$env:LOCALAPPDATA\ai-agents-skills\runtime" }
& "$runtime\run_skill.ps1" "skills/kaggle-research-compute/run_kaggle_research_compute.ps1" <args>
```

POSIX examples below use `run_skill.sh` and `.sh` command targets; use the Windows command target above on native Windows.

Use this skill when the task is about:

- exhaustive search
- object enumeration
- counterexample hunting
- large parameter sweeps
- long-running CPU or GPU batch work that a throttled local run cannot finish in time

This skill is the Kaggle Kernels lane of the local `research_compute` broker. It packages a
portable job bundle as a kernel, pushes it, polls its status, and downloads its output, and it
runs a multi-run resume loop across concurrent kernels for jobs that need more than one 12h
session. It is peer to the Modal, Hetzner, and GitHub Actions lanes.

## When to prefer this skill

- the workload is CPU-heavy batch work: Kaggle CPU is FREE and does NOT consume the GPU quota, so it is preferred over the paid/quota'd lanes for any CPU job that fits Kaggle's constraints
- the workload wants a GPU and fits within the self-imposed weekly GPU-hour cap (Kaggle GPU is free under the ~30h/week floating quota)
- the job is chunkable and resumable to at most a 12h session per kernel run on ~4 vCPU / ~32 GB
- routing order is `local > Kaggle > Modal > Hetzner > GitHub Actions`, so Kaggle is the FIRST offload tier (right behind local) whenever credentials are present and the job fits

## Unified routing

The umbrella doc `compute-offload-routing.md` explains backend selection across the five lanes
(local, Kaggle, Modal, Hetzner, GitHub Actions), the keep-local rules, and the local
self-preservation veto. The per-lane contract for Kaggle — driver verbs, the multi-run resume
loop, the concurrency fan-out, the free-CPU / weekly-GPU-cap model, and guardrails — is in
`references/kaggle-offload.md`. The broker router is the decision boundary: `plan` and `doctor`
choose the backend; this skill pushes kernels only after that choice lands on Kaggle.

## Core workflow

Work through `compute-offload-sizing-gate` first. Size against the lane's
**aggregate** capacity — `kernel_cores` x `concurrency`, not one kernel — and
set `total_units` so the fan-out is actually used; sizing to a single kernel
understates the free lane by the concurrency factor. `preflight` reports
`kernel_cores`, `kernel_ram_gb` and `aggregate_cores` alongside `est_kernels`
and `est_rounds`, which are derived from `total_units` as well as `core_hours`,
so the estimate matches the kernels the run loop will actually launch.

1. If local resources matter, run `get-available-resources` and let the broker apply the self-preservation veto.
2. Build a portable job bundle (`manifest.json` with `total_units`, `worker`, `run.sh`, `merge`, writable `out/`) — the same bundle runs unchanged on any lane; each completed work unit leaves a checkpoint in `out/` so a re-pushed kernel resumes.
3. Run `preflight` (free, no kernel) to get the Kaggle plan: kind (CPU/GPU), estimated resume rounds and kernel count, concurrency, the 12h session cap, the GPU-hour estimate vs the weekly cap, adequacy, and availability.
4. If the plan fits, `run` executes the multi-run resume loop: it fans out up to ~5 concurrent kernels per round, polls them, fetches checkpoints, and re-pushes the remaining work with the checkpoints re-attached until the job is DONE (bounded by `max_runs`).
5. For manual control, `push` one kernel (a chunk), `status`/`wait` to poll it, and `fetch` to download its output.
6. No teardown: kernels auto-stop at the 12h session cap and cost nothing, so there is no reaper and nothing to destroy.

## Runtime commands

Linux (resolve the launcher for the current agent — a root-owned component generation when one is installed, otherwise the per-user runtime — then call it):

```bash
# Credential-bearing lanes must launch from a root-owned AAS component
# generation; the per-user runtime copy is refused by the credential gate.
# The generation directory is named for a git commit, so its name carries no
# ordering -- pick the newest publish, which the store records as its mtime.
launcher="${AAS_RUNTIME_ROOT:-$HOME/.local/share/ai-agents-skills/runtime}/run_skill.sh"
newest=0
for gen in /usr/local/libexec/coding-system/components/ai-agents-skills/*/; do
  gen="${gen%/}"
  [ -f "$gen/manifest/credential-runtime.json" ] || continue
  [ -x "$gen/canonical/runtime/runners/run_skill.sh" ] || continue
  stamp="$(stat -c %Y "$gen" 2>/dev/null || stat -f %m "$gen" 2>/dev/null)" || continue
  [ "${stamp:-0}" -gt "$newest" ] || continue
  newest="$stamp"
  launcher="$gen/canonical/runtime/runners/run_skill.sh"
done
run() { bash "$launcher" skills/kaggle-research-compute/run_kaggle_research_compute.sh "$@"; }
```

```bash
run bootstrap                          # one-time: check kaggle CLI + kagglehub + API token, validate via kagglehub, run doctor
run doctor                             # lane + credentials + kaggle CLI + configured caps (offline)
run preflight --job /path/to/jobdir --json     # the plan the router consumes (no kernel)
run run     --job /path/to/jobdir --confirm    # multi-run resume loop with concurrent fan-out until DONE
run push    --job /path/to/jobdir --confirm    # push one kernel run (a chunk), for manual control
run status  <user/kernel-slug>
run wait    <user/kernel-slug>
run fetch   <user/kernel-slug> --dest /path/to/output
```

Planning verbs (`bootstrap`, `doctor`, `preflight`) are free and never push a kernel.
Lifecycle verbs (`push`, `status`, `wait`, `fetch`, `run`) submit real kernels and require the
new Kaggle API token (`KAGGLE_API_TOKEN`, or `~/.kaggle/access_token`) plus an explicit
`--confirm`. Use `--dry-run` on `push` and `run` to print the exact planned `kaggle` commands
with nothing submitted.

On targets that install a local skill wrapper, that wrapper should forward to the same
runtime command target.

```bash
skills/kaggle-research-compute/run_kaggle_research_compute.sh doctor
```

Windows:

```powershell
$runtime = if ($env:AAS_RUNTIME_ROOT) { $env:AAS_RUNTIME_ROOT } else { "$env:LOCALAPPDATA\ai-agents-skills\runtime" }
& "$runtime\run_skill.ps1" `
  "skills/kaggle-research-compute/run_kaggle_research_compute.ps1" `
  doctor
```

## Operational notes

- The broker is the decision boundary. Push kernels on Kaggle only when the router chose this lane.
- Auth is the new single Kaggle API token, read from `KAGGLE_API_TOKEN` (or `~/.kaggle/access_token`) at runtime (env-first, never argv, never logged) — NOT the legacy `KAGGLE_USERNAME` + `KAGGLE_KEY` pair. `bootstrap` validates/primes via kagglehub (`kagglehub.whoami()` proves the token and yields the username the kaggle CLI uses for kernel ops). Do not write a `kaggle.json` into the repo or a kernel; a redaction filter covers surfaced output.
- When `AAS_COMPUTE_SECRETS_FILE` names the shared protected compute authority,
  the managed wrapper validates its full schema but projects only
  `KAGGLE_API_TOKEN` and `KAGGLE_CONFIG_DIR`; Hetzner values and the pointer are
  removed before the Kaggle child starts.
- Caps live under `[kaggle]` in `research-compute.toml`: `weekly_gpu_hours_cap`, `max_runs`, `concurrency`, `session_hours`, and the free-tier `kernel_cores` / `kernel_ram_gb`. CPU work is free and quota-free; GPU work passes a fail-closed weekly GPU-hour gate that reserves the estimate in a local usage ledger before the first push, so concurrent GPU submits cannot collectively blow the weekly cap.
- The 12h session cap is the reason for the multi-run resume loop. A job that needs more wall time spans multiple kernel runs: push a chunk-batch across up to ~5 concurrent kernels, download checkpoints, and re-push the remaining chunks with the checkpoints re-attached (as a Kaggle Dataset input) until DONE. `max_runs` bounds the loop.
- No reaper, no dead-man's-switch, no teardown: kernels auto-stop at the 12h session cap and cost nothing, so this lane is materially lower-risk than a paid rented-server lane. There is no cost gate — Kaggle is free.
- `doctor` and `preflight` work without a token and without a kernel. `push`, `status`, `wait`, `fetch`, and `run` need the host to be Kaggle-ready (the `kaggle` CLI >=1.8.0 and kagglehub >=0.4.1 installed and `KAGGLE_API_TOKEN` set, or `~/.kaggle/access_token` present).
- One-time per machine, run `bootstrap`: it checks the `kaggle` CLI and kagglehub, confirms the API token is present, and validates/primes via kagglehub (`kagglehub.whoami()`), then reports `doctor`. It never pushes a kernel.
- ToS: Kaggle compute is intended for its data-science / competition platform. Keep to modest, legitimate research workloads and verify the current Kaggle terms permit this use before the first live run. The build and its tests make no live Kaggle calls.

## Recommended templates

When this skill is involved, consider the same workflow templates as the other offload lanes
(install via the `workflow-templates` artifact profile, or `--with-deps` to pull backing skills):

- `compute-offload-sizing-gate` -- Pre-dispatch worksheet: measure the workload, read the declared lane capacity, write the manifest in the correct dialect, assert the plan, and verify the realized allocation.
- `autonomous-research-loop-runbook` -- Bounded autonomous research-loop runbook with four stop conditions, single-path solving, mandatory cross-agent verification, fresh-agent backtracking, and five-lane broker-routed heavy-compute offload with per-lane safety gates.
- `engineering-delivery-loop-runbook` -- Bounded build-and-deliver loop runbook: single-path implementation with seen-to-fail proof, cross-agent diff verification, behavior-preserving cleanup, and five-lane broker-routed heavy-compute offload with per-lane safety gates.