# Path Storage Policy

> Enforce a general path planning and storage policy: keep /home for code, configs, manifests, and standard lightweight experiment results, while placing caches, environments, datasets, model weights, checkpoints, intermediate artifacts, and any large files under /scratch. Use when deciding where to create files, store outputs, download assets, configure caches, or lay out experiment directories.

- Skill: `kwongfuk/path-storage-policy` (Agent Skill)
- Install (CLI): `npx skillmds@latest add kwongfuk/path-storage-policy`
- Raw SKILL.md: https://api.skillmd.com/api/skills/kwongfuk/path-storage-policy/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: KwongFuk (https://skillmd.com/u/kwongfuk)
- Updated: 2026-09-22
- Page: https://skillmd.com/skills/kwongfuk/path-storage-policy

---


# Path Storage Policy

Apply this skill whenever a task creates, downloads, writes, moves, or plans files and directories.

## Core Policy

- Treat `/home/$USER` as the durable source tree area.
- Treat `/scratch/$USER` as the heavy-storage and high-churn area.
- Keep code, configs, notebooks, manifests, small logs, and standard lightweight experiment results under `/home/$USER`.
- Keep environments, caches, datasets, model weights, checkpoints, container images, raw intermediates, append-heavy outputs, and any large file or large directory under `/scratch/$USER`.
- If an artifact may grow substantially, is expensive to regenerate, or may exceed home quota, place it on scratch by default.
- If only a compact final summary is needed, keep the summary in the project tree and leave the heavy upstream artifacts on scratch.

## Decision Rules

1. Put it in `/home/$USER` when it is primarily:
- source code
- configuration
- notebooks
- lightweight documentation
- small manifests or metadata
- standard small experiment outputs such as final tables, plots, and compact metrics

2. Put it in `/scratch/$USER` when it is primarily:
- a cache
- a virtual environment or Conda environment
- downloaded data
- downloaded or generated model weights
- checkpoints or resumable run state
- raw predictions, intermediate outputs, or staged preprocessing results
- a container image
- a large result directory
- a file or directory that is large, append-heavy, temporary-but-costly, or high churn

3. If uncertain, prefer this split:
- small human-reviewed deliverables in `/home/$USER`
- heavy machine-generated artifacts in `/scratch/$USER`

## Planning Workflow

1. Identify every path the task will read or write.
2. Classify each path as one of:
- source/config
- cache/env
- dataset/model
- checkpoint/intermediate
- final result
- log/record
3. Place source/config, logs, records, and small standard outputs under `/home/$USER`.
4. Place cache/env, dataset/model, checkpoints, intermediates, and large outputs under `/scratch/$USER`.
5. When a final result has both heavy and lightweight forms, store:
- heavy primary artifact on scratch
- lightweight summary or pointer in the project tree
6. Use absolute paths when the runtime environment may rewrite `$USER` or `$HOME`.

## Recommended Layout

```text
/home/$USER/code/<project>/
/home/$USER/code/<project>/configs/
/home/$USER/code/<project>/notebooks/
/home/$USER/code/<project>/results/
/home/$USER/code/<project>/logs/

/scratch/$USER/envs/
/scratch/$USER/.cache/
/scratch/$USER/data/
/scratch/$USER/models/
/scratch/$USER/containers/
/scratch/$USER/runs/<project>/
/scratch/$USER/checkpoints/<project>/
```

## Operational Rules

- Before downloads or installs, redirect caches to scratch.
- Before long runs, ensure checkpoint and large-output directories point to scratch.
- Do not silently write large artifacts under `/home/$USER`.
- If a tool defaults to `~/.cache`, `~/.conda`, or project-local heavy outputs, override it to scratch when practical.
- If the task is on shared compute or HPC, assume scratch is the default destination for heavy artifacts unless the user explicitly asks otherwise.

## Examples

- A git repository, training script, and final markdown report belong under `/home/$USER`.
- A Conda env, Hugging Face cache, dataset shard, and model checkpoint belong under `/scratch/$USER`.
- A 200 MB prediction dump belongs under `/scratch/$USER`, while a 20 KB summary CSV can stay in the repository.
- A long-running experiment may keep compact metrics under the project `results/` directory while storing checkpoints and raw batch outputs on scratch.