# Foundations Bash Scripting

> Write robust Bash scripts to batch-process FASTQ/BAM/VCF/FASTA files: variables, set -euo pipefail error handling, loops over sample sheets, functions, traps, and awk/sed text processing. Use when automating a multi-sample pipeline, writing a shell wrapper around samtools/bcftools/fastqc/blast, validating CLI input files, or debugging a script that fails silently or mishandles filenames with spaces.

- Skill: `pavel-kravchenko/foundations-bash-scripting` (Agent Skill)
- Install (CLI): `npx skillmds@latest add pavel-kravchenko/foundations-bash-scripting`
- Raw SKILL.md: https://api.skillmd.com/api/skills/pavel-kravchenko/foundations-bash-scripting/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: DevOps & Infra
- Author: pavel-kravchenko (https://skillmd.com/u/pavel-kravchenko)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/pavel-kravchenko/foundations-bash-scripting

---


# Bash Scripting for Bioinformatics

## When to Use

- Automating the same command (FastQC, samtools, BLAST, alignment) over dozens–hundreds of samples.
- Writing the shell glue that Snakemake/Nextflow rules or a cron job calls under the hood.
- Building a CLI wrapper script that validates input files/arguments before running an expensive tool.
- Debugging a pipeline step that "worked once" but silently produced empty/wrong output (missing `set -euo pipefail`, unquoted variables, unmatched glob).
- Generating a sample sheet or summary report from a directory of FASTQ/BAM/VCF files with `awk`/`sed`/`cut`.

## Version Compatibility

- GNU Bash ≥ 4.4 (associative arrays, `${var,,}`); Bash 5.x on most modern Linux distros. macOS ships Bash 3.2 — install Bash 5 via Homebrew if targeting that.
- Coreutils/`awk`/`sed`/`grep` as found on any standard Linux distro (GNU variants; BSD `sed`/`awk` on macOS differ slightly, e.g. `sed -i ''`).

## Prerequisites

- Comfortable with the Linux command line (`foundations-linux-fundamentals`): paths, pipes, redirection, `grep`/`cut`/`sort`.
- No extra packages to install — Bash and coreutils are present on any POSIX system. Tool-specific commands shown below (`fastqc`, `samtools`, `bcftools`, `blastn`) assume those tools are separately installed and on `$PATH`.

**Goal:** produce a script that never silently continues after a failure and never breaks on filenames containing spaces.
**Approach:** start every script with strict-mode + a logger, validate inputs before doing real work, then loop.

```bash
#!/bin/bash
set -euo pipefail
# -e: exit on any non-zero command; -u: error on unset variables (catches typos);
# -o pipefail: a failing command inside a pipe fails the whole pipe.

THREADS=8
INPUT_DIR="${1:?Usage: $0 <input_dir> <output_dir>}"
OUTPUT_DIR="${2:?Usage: $0 <input_dir> <output_dir>}"

log() { echo "[$(date '+%H:%M:%S')] $1" >&2; }

require_tool() {
    # Fail fast if a needed binary isn't on PATH.
    local tool="$1"
    command -v "$tool" &>/dev/null || { log "ERROR: $tool not found"; exit 1; }
}

[[ -d "$INPUT_DIR" ]] || { log "ERROR: not a directory: $INPUT_DIR"; exit 1; }
mkdir -p "$OUTPUT_DIR"
require_tool fastqc
log "Starting..."
```

**Goal:** batch-process every FASTQ pair in a directory without breaking on an empty glob or a missing mate.
**Approach:** glob + guard, derive R2 from R1 with pattern substitution, skip (don't crash) on missing pairs.

```bash
#!/bin/bash
set -euo pipefail

INPUT_DIR="${1:-.}"
OUTPUT_DIR="${2:-qc_results}"
mkdir -p "$OUTPUT_DIR"

count=0
for r1 in "${INPUT_DIR}"/*_R1.fastq.gz; do
    [[ -f "$r1" ]] || { echo "No *_R1.fastq.gz files found in $INPUT_DIR" >&2; exit 1; }

    r2="${r1/_R1/_R2}"                       # pattern substitution: _R1 -> _R2
    sample=$(basename "$r1" _R1.fastq.gz)

    if [[ ! -f "$r2" ]]; then
        echo "WARNING: missing R2 for $sample, skipping" >&2
        continue
    fi

    echo "[$((++count))] fastqc -t 4 -o \"$OUTPUT_DIR\" \"$r1\" \"$r2\""
    # fastqc -t 4 -o "$OUTPUT_DIR" "$r1" "$r2"
done
echo "Processed $count pairs"
```

**Goal:** read a tab-separated sample sheet and summarize gene counts, without touching Python.
**Approach:** `while IFS=$'\t' read -r ...` for line-oriented parsing; `awk -F'\t'` for column math.

```bash
#!/bin/bash
set -euo pipefail

# Sample sheet: sample_id<TAB>condition
while IFS=$'\t' read -r sample_id condition; do
    echo "Sample: $sample_id | Condition: $condition"
done < sample_sheet.tsv

# Genes with mean count across 3 columns > 1000, from a TSV with a header row
awk -F'\t' 'NR>1 { avg=($2+$3+$4)/3; if (avg>1000) print $1, "avg="int(avg) }' gene_counts.tsv
```

Case statement for routing by file type:

```bash
case "$file" in
    *.fastq.gz | *.fq.gz) fastqc "$file" ;;
    *.bam)                samtools flagstat "$file" ;;
    *.vcf | *.vcf.gz)     bcftools stats "$file" ;;
    *.fasta | *.fa)       grep -c "^>" "$file" ;;
    *) echo "Unknown: $file" >&2; exit 1 ;;
esac
```

Cleanup on exit (temp dirs, partial output) with `trap`:

```bash
TMPDIR=$(mktemp -d)
cleanup() { rm -rf "$TMPDIR"; }
trap cleanup EXIT     # runs on normal exit AND on error (with set -e)
```

## Comparison Table

| Goal | Command |
|------|---------|
| Default value | `${var:-default}` |
| Require arg | `${1:?Usage: ...}` |
| Strip suffix | `${var%.fastq.gz}` |
| Strip extension via basename | `$(basename "$f" .gz)` |
| Dir of file | `$(dirname "$f")` |
| Count lines | `wc -l < "$file"` |
| Redirect stderr | `cmd 2>/dev/null` |
| N-way parallel | `find . -name '*.gz' \| xargs -P4 -I{} fastqc {}` |

## Pitfalls

- **`set -euo pipefail` is non-negotiable**: without it, a failed command inside a pipe (`cat missing | wc -l`) reports the exit code of `wc`, not `cat`, and the script continues on garbage.
- **No spaces around `=`**: `var=value` is correct; `var = value` runs a command named `var`.
- **Always double-quote variables**: `"$var"` not `$var` — a filename with a space breaks unquoted expansion into multiple words/arguments.
- **`$()` not backticks**: backticks cannot be nested and are harder to read.
- **`local` in functions**: undeclared variables leak into global scope and can clobber an outer variable with the same name.
- **Glob matching zero files**: `for f in dir/*.gz` — when nothing matches, the loop body still runs once with the literal string `dir/*.gz`; always guard with `[[ -f "$f" ]] || { ...; exit 1; }` at the top of the loop.
- **`-u` plus optional args**: with `set -u`, referencing `$1` when no argument was passed is a hard error — use `${1:-default}` or `${1:?message}` instead of bare `$1`.

## See Also

- `foundations-linux-fundamentals` — command-line basics this skill builds on.
- `bio-workflow-management-snakemake-workflows` — when a script grows past a few pipeline steps, wrap it in Snakemake instead of chaining more Bash.
- `bio-workflow-management-nextflow-pipelines` — alternative workflow manager for multi-step, multi-sample pipelines.
- `foundations-statistics-python` — once data leaves the shell (counts, tables), continue analysis in Python/pandas.