Qmd Runner

QMD Runner Debugging

Overview

• Problem: quarto render aborts at the first chunk that errors, so a single broken cell hides every downstream failure. Iterating one error at a time on a 20-chunk analysis notebook is slow.
• Solution: programmatically extract each {r} chunk into a sibling .R runner where every chunk is wrapped in tryCatch. Run the runner, get a single PASS / FAIL / SKIP table for every chunk in one pass, fix everything, repeat.
• Rule: every fix lands in the source .qmd. The runner is regenerated from the qmd each iteration — never hand-edit the runner.
• End state: when every chunk is PASS / SKIP, the corresponding quarto render will succeed too. Renders go from a debugging tool to a final-output tool.

When to Use

• A quarto render of an analysis notebook errors out and the message is hard to parse
• Adopting an inherited pipeline that has not been re-rendered in months
• Upgrading a key dependency (Seurat, dplyr, msigdbr, biomaRt, clusterProfiler) and a sweep of qmds needs revalidating
• Many chunks share in-memory state (e.g. GSEA pipelines where feature_loadings flows through several PC chunks) and you want chunk-level isolation of failures
• Multi-script pipelines where downstream notebooks read checkpoints from upstream notebooks (creating-analysis-projects read → scripts → checkpoints → write layout) and you want to validate the whole DAG

When NOT to Use

• Single-chunk qmds where the cost of building a runner exceeds the diagnostic gain — just quarto render directly
• Render failures caused by YAML / front-matter errors (not chunk code) — those need a different fix
• Producing the polished .html artefact — the runner does not invoke Quarto, so the html does not get rebuilt; finish with quarto render once chunks are green

Workflow

R0 — Wire up the generator (once per repo)

• Place runners/generate-runner.R at the repo root and a matching .gitignore exclusion
• The generator is the only thing that gets committed; per-script runner outputs stay gitignored

.gitignore excerpt:

# Per-chunk debug runners (auto-generated from qmds)
!runners/
!runners/generate-runner.R
mcf7/scripts/runners/
hcc1806/scripts/runners/

R1 — Generate runners for every active qmd

• Run Rscript runners/generate-runner.R <qmd-path> (or pass several) for every notebook to be debugged
• Each emits <dirname(qmd)>/runners/<basename-without-ext>.R
• Skip notebooks that are not worth runner-debugging (e.g. credentials-only deployment scripts)

R2 — Iterative debug loop

For each runner, in dependency order (01 → 02 → 03 → ...):

1. Rscript <runner>.R 2>&1 | tee <runner>.log
2. Read the PASS / FAIL / SKIP table at the bottom
3. For each FAIL: a. Diagnose the underlying error from the runner's Error: line b. Edit the source qmd (not the runner) to fix the root cause c. Rscript runners/generate-runner.R <qmd-path> to regenerate d. Re-run, confirm the chunk now PASSes
4. Move to the next runner once every chunk in the current one is PASS or SKIP

R3 — Commit the fixes

• git status now shows only the .qmd edits from step R2
• Commit with a single conventional-commits message (e.g. fix: address chunk-level errors surfaced by runners), or split per-script if the diff is large
• Runners stay untracked

R4 — Final quarto render

• Run quarto render <qmd> sequentially in dependency order to produce the polished html and repopulate write/ + checkpoints/ end-to-end
• All chunks should pass since the runners already proved it; if any chunk does fail in the actual render, it is usually a render-time concern (e.g. knitr cache, fig device, working directory) — drop back into R2 for that script

R5 — Report

• Print a per-script PASS table
• Verify checkpoints and figures landed where expected

Anatomy of a Runner

Generated runner preamble (every runner starts with this):

# Locate the project .Rprofile by walking up from getwd() until we find it,
# then setwd(repo_root) BEFORE sourcing it so renv/activate.R picks the
# correct project library path. Finally setwd() into the script's directory
# so the qmd's relative paths (../checkpoints/, ../write/) resolve correctly.
local({
  cand <- normalizePath(getwd(), winslash = "/")
  for (i in 1:8) {
    if (file.exists(file.path(cand, ".Rprofile")) && file.exists(file.path(cand, "renv/activate.R"))) break
    parent <- dirname(cand)
    if (parent == cand) break
    cand <- parent
  }
  Sys.setenv(SMARTSEQ_REPO_ROOT = cand)
  prev_wd <- getwd()
  setwd(cand)
  source(".Rprofile", local = FALSE)
  setwd(prev_wd)
})
setwd(file.path(Sys.getenv("SMARTSEQ_REPO_ROOT"), "<cell-line>/scripts"))

# Chunk-level outcome accumulator.
RUNNER_RESULTS <- list()

run_chunk <- function(label, expr) {
  cat(sprintf("\n=== [%s] START: %s ===\n", format(Sys.time(), "%H:%M:%S"), label))
  result <- tryCatch({
    eval(expr, envir = .GlobalEnv)
    cat(sprintf("=== [%s] PASS:  %s ===\n", format(Sys.time(), "%H:%M:%S"), label))
    list(status = "PASS", error = NA_character_)
  }, error = function(e) {
    cat(sprintf("=== [%s] FAIL:  %s ===\n", format(Sys.time(), "%H:%M:%S"), label))
    cat(sprintf("  Error: %s\n", conditionMessage(e)))
    list(status = "FAIL", error = conditionMessage(e))
  })
  RUNNER_RESULTS[[label]] <<- result
  invisible(result)
}

skip_chunk <- function(label, reason = "marked eval: false in qmd") {
  cat(sprintf("\n=== [%s] SKIP:  %s (%s) ===\n", format(Sys.time(), "%H:%M:%S"), label, reason))
  RUNNER_RESULTS[[label]] <<- list(status = "SKIP", error = NA_character_)
  invisible(NULL)
}

Each chunk in the runner becomes:

# ---- chunk: <label> (qmd line ~<N>) ----
run_chunk("<label>", quote({
  <chunk body>
}))

For chunks marked #| eval: false in the qmd:

# ---- chunk: <label> ----
skip_chunk("<label>")

Summary block at the bottom prints the PASS / FAIL / SKIP table and quit(status = 1L) if any FAIL.

The Generator

runners/generate-runner.R is responsible for:

• Reading the qmd line-by-line
• Detecting {r} chunk opens ( ```{r} or ```{r ...}) and ``` closes
• Lifting #| label: and #| eval: options out of each chunk header
• Indenting the chunk body two spaces and wrapping it in run_chunk(<label>, quote({ ... }))
• Emitting one file at <dirname(qmd)>/runners/<basename-without-ext>.R
• Being idempotent: re-running overwrites without prompting

Why eval in .GlobalEnv: Quarto / knitr renders each chunk into the document's global environment, so any chunk-to-chunk state-sharing (functions defined in chunk 1 used in chunk 5) Just Works. The runner mirrors this exactly so a passing runner predicts a passing render.

Run the generator on multiple qmds at once:

Rscript runners/generate-runner.R \
  mcf7/scripts/01-processing-pipeline.qmd \
  mcf7/scripts/02-tables-visualisations.qmd \
  hcc1806/scripts/01-processing-pipeline.qmd

Common Failure Patterns

These come up over and over on real-world Seurat / Bioconductor pipelines. Fix them in the source .qmd, not the runner.

biomaRt mirror-fallback + cache pattern (the universal fix for the Ensembl errors):

annotation_cache <- "../checkpoints/<NN>-<cellline>-biomart-annotation.rds"
if (file.exists(annotation_cache)) {
  annotation <- read_rds(annotation_cache)
} else {
  human <- NULL
  for (mirror in c("useast", "asia", "www")) {
    human <- tryCatch(
      useEnsembl(biomart = "genes", dataset = "hsapiens_gene_ensembl", mirror = mirror),
      error = function(e) NULL
    )
    if (!is.null(human)) break
  }
  if (is.null(human)) stop("biomaRt: every Ensembl mirror failed to respond")
  annotation <- getBM(
    attributes = c("ensembl_gene_id", "hgnc_symbol"),
    filters    = "ensembl_gene_id",
    values     = ids_to_lookup,
    mart       = human
  )
  write_rds(annotation, annotation_cache)
}

Dependency Ordering

When a project has multiple notebooks that pass data via checkpoints, run runners in upstream-to-downstream order:

• Root notebooks first (data import, QC, normalisation) — they produce the canonical clustered checkpoint
• Then notebooks that read that checkpoint (visualisation, DEA, enrichment)
• Then notebooks that read DEA results (paper figures, target curation, app exports)

Pattern: 01 → 02 / 03 → 04+ in parallel. Catching a 01 failure early prevents 12 cascading downstream failures.

Cross-line projects (e.g. MCF7 vs HCC1806) often vendor an upstream checkpoint into a sibling tree's read/reference/external/. Re-vendor that checkpoint immediately after the upstream notebook succeeds so downstream cross-line scripts pick up the fresh version:

cp hcc1806/checkpoints/01-hcc1806-seurat-clustered.rds \
   mcf7/read/reference/external/02-hcc1806-seurat-clustered.rds

Tracked vs Gitignored

• Tracked: runners/generate-runner.R (the generator itself) and the .gitignore rules
• Gitignored: every <script-dir>/runners/<file>.R and <file>.log — these are derivative

Pairs Well With

• writing-r-code — the per-chunk Libraries / Inputs / Processing / Outputs structure that this debugging loop assumes
• writing-qmd-scientific — hashpipe (#|) chunk options that the generator parses
• creating-analysis-projects — checkpoint-flow conventions that explain why downstream chunks fail when an upstream notebook has not been re-run
• git-hygiene — conventional commit format for the R3 fix-commits