Research Project Layout

Research Project Layout

Use this skill when creating, moving, saving, or reorganizing files in research repositories. Follow the AuroraLF-style convention: separate files by ownership, reproducibility role, and expected lifetime.

Top-level layout

• Put the repository's primary maintained Python package at the top level, named after the project when one exists, e.g. auroralf/ or smallscale21cm/.
• Put maintained local packages in packages/.
  • Use this for code the project imports, tests, and edits directly, such as a refactored local EoRCaLC.
  • Do not put unmodified external source trees here.
• Put executable project workflows in scripts/.
  • scripts/run/: project run entrypoints.
  • scripts/submit/: SLURM or batch submission scripts.
  • scripts/analysis/: reproducible analysis scripts that read project data and make derived products.
  • scripts/plot/: reusable plotting scripts when a repo already separates plotting from analysis.
  • scripts/experiments/: exploratory or legacy scripts that are not yet stable APIs.
• Put notebooks in notebooks/<topic>/, grouped by scientific task or subsystem.
• Put small public or literature data in external_data/.
  • Use external_data/observations/ for observational constraints and published tables.
  • Use descriptive subdirectories such as external_data/literature_sources/, external_data/empirical_models/, or external_data/ssp_spectra/ when appropriate.
  • Do not put third-party source repositories here.
• Put external source trees and vendored dependencies in third_party/.
  • These are not the project's maintained packages.
  • Keep large third-party trees ignored by default unless the user explicitly wants to version them.
• Put large simulation and run directories in runs/.
  • Use this for raw fields, run-specific caches, bulk SLURM output, and directories that can reach GB/TB scale.
  • Keep runs/ ignored by default.
• Put reusable project-generated data products in data_save/.
  • Use for summary tables, .csv, .tsv, .npz, and expensive intermediate results that will be reused.
• Put temporary or diagnostic products in outputs/.
  • Use for logs, quick-look figures, progress files, one-off diagnostics, and throwaway scripts created during investigation.
• Put paper materials in manuscript/.
  • Use for manuscript source, publication PDFs, revision responses, bibliography, and paper-local assets.
• Put formal presentation materials in slides/.
  • Use for slide source, final slide PDFs, and slide-dependent assets.
• Put tests in tests/ unless the project or local package already has a more specific test location.

Decision rules

• If the project owns and maintains code, put it in the primary package or packages/.
• If code is an external upstream project, put it in third_party/.
• If data comes from observations, papers, public catalogs, or external model releases, put it in external_data/.
• If data is generated by this project and reused later, put it in data_save/.
• If data is a large raw run product or run-local cache, put it in runs/.
• If a figure or log is for inspection, debugging, or a one-off check, put it in outputs/.
• If a figure is required to compile slides, put the slide copy in slides/assets/.
• If a figure is required only by a manuscript, keep it in manuscript/ or a manuscript-local asset directory.

Slides

• If generating or updating slide decks, write the source and final PDF to slides/.
• If a LaTeX slide depends on figures, keep those figures in slides/ as well, preferably under slides/assets/.
• Slide figure assets should be publication-quality.
  • Prefer vector .pdf assets for figures inserted into slides.
  • Do not add .png or other raster figures to slides/assets/ unless the user explicitly asks for that format.
  • When a raster export is also needed for preview or sharing, save it outside slides/assets/ and use at least 500 dpi.
• Keep slides/assets/ minimal.
  • Retain only assets that are actively referenced by the current slide source(s) in slides/.
  • If an image is replaced, deprecated, or no longer referenced by any slide .tex, remove it from slides/assets/.
  • Do not use slides/assets/ as a cache for old draft figures; keep draft or one-off figures in outputs/ instead.

When modifying scripts

• Prefer setting output paths so future runs follow this layout automatically.
• Convert hard-coded relative paths such as ./output, ./fig, ./reionf, or old project names into explicit project-root paths.
• Keep scientific behavior unchanged unless the user asked for a modeling change.
• Do not add fallback paths to old layouts; update the real path and let missing files fail clearly.
• When a slide currently points at images in outputs/, move or copy the required compile-time assets into slides/assets/ and update the slide source.
• When preparing figures for slide insertion, generate and reference a .pdf asset in slides/assets/.
• If you also make a raster companion image for checking or chat display, export it at >=500 dpi and keep it in outputs/ unless the user asks otherwise.
• After updating slide assets, check which files are still referenced by slides/*.tex and prune stale files from slides/assets/.
• Avoid adding project-local README files just to describe these storage rules.

Git hygiene

• Track source code, small configs, tests, manuscripts when appropriate, and curated small external data.
• Ignore large generated data, run outputs, caches, local environments, temporary figures, and third-party source trees unless the user explicitly wants them versioned.
• Before committing a layout change, check git status --short --ignored so ignored large directories and unignored curated data match the intended boundary.