Sed

GNU sed Skill

All output assumes GNU sed (sed --version shows GNU). POSIX compatibility is not a goal; use -E (extended regex) by default unless basic regex is clearly sufficient.

Core Principles

1. Prefer one-liners. Multi-line sed scripts are a last resort.
2. Use -E for extended regex (avoids backslash noise).
3. Use -i for in-place edits; always show the command with a backup suffix suggestion (-i.bak) if the operation is destructive.
4. Dry-run first. When editing files, show the sed without -i before the in-place version so the user can verify.
5. Chain with pipes when sed alone isn't the cleanest tool.
6. Never use sed to parse structured formats (JSON, YAML, XML). Redirect to jq, yq, or xmllint and say why.

Flags to Always Know

Command Quick Reference

Substitution (s)

# Basic replace (first occurrence per line)
sed 's/foo/bar/'

# Replace all occurrences on each line
sed 's/foo/bar/g'

# Case-insensitive replace (GNU extension)
sed 's/foo/bar/gI'

# Replace only on lines matching a pattern
sed '/error/s/foo/bar/g'

# Replace on line ranges
sed '10,20s/foo/bar/g'

# Use & to reference the matched text
sed 's/[0-9]\+/(&)/g'   # wraps every number in parens

# Capture groups with \1, \2
sed -E 's/(foo)(bar)/\2\1/'

# Delimiter swap (useful when pattern contains /)
sed 's|/usr/local|/opt|g'

Delete lines (d)

# Delete lines matching a pattern
sed '/^#/d'

# Delete blank lines
sed '/^[[:space:]]*$/d'

# Delete line range
sed '5,10d'

# Delete from pattern to end of file
sed '/START/,$d'

# Delete lines NOT matching (invert with !)
sed '/keep/!d'

Print / Extract (p, -n)

# Print only matching lines (like grep)
sed -n '/error/p'

# Print line numbers with matches
sed -n '/error/='

# Print a line range
sed -n '10,20p'

# Print from pattern to pattern (inclusive)
sed -n '/START/,/END/p'

# Print only the captured group (GNU extension)
sed -En 's/.*key=([^ ]+).*/\1/p'

Insert / Append / Change (i, a, c)

# Insert a line before match
sed '/pattern/i\NEW LINE'

# Append a line after match
sed '/pattern/a\NEW LINE'

# Replace the entire matching line
sed '/pattern/c\REPLACEMENT LINE'

Transform characters (y)

# Like tr: map chars one-to-one
sed 'y/abc/ABC/'

Multi-expression

sed -E -e 's/foo/bar/g' -e 's/baz/qux/g'

Patterns & Addresses

/regex/         — lines matching regex
/re1/,/re2/     — line range from re1 to re2 (inclusive)
N               — line number N
N,M             — line numbers N through M
N~STEP          — every STEP lines starting at N  (e.g. 1~2 = odd lines)
$               — last line
/re/!           — lines NOT matching

Common Use Cases

Shell scripting: rewrite config values

# Dry-run
sed -E 's/^(TIMEOUT=).*/\1120/' config.env

# In-place with backup
sed -E -i.bak 's/^(TIMEOUT=).*/\1120/' config.env

Code refactoring: rename symbol across files

# Preview
grep -rl 'OldClass' src/ | xargs sed -E 's/\bOldClass\b/NewClass/g'

# In-place
grep -rl 'OldClass' src/ | xargs sed -E -i.bak 's/\bOldClass\b/NewClass/g'

Use \b for word boundaries (GNU sed supports \b in ERE).

Log processing: extract fields

# Extract timestamps from nginx logs
sed -En 's/.*\[([^\]]+)\].*/\1/p' access.log

# Show only lines between two timestamps (approximate)
sed -n '/2024-01-01 10:00/,/2024-01-01 11:00/p' app.log

# Strip ANSI color codes
sed -E 's/\x1B\[[0-9;]*[mK]//g'

File batch editing: add/remove lines

# Add a shebang to all .sh files missing one
find . -name '*.sh' | xargs sed -i '1{/^#!/!i\#!/usr/bin/env bash
}'

# Remove trailing whitespace
sed -E -i 's/[[:space:]]+$//' file.rb

# Comment out lines matching a pattern
sed -i 's/^FEATURE_X=/#FEATURE_X=/' .env

Delete lines between markers (exclusive)

sed '/START/,/END/{/START/!{/END/!d}}' file.txt

Number lines (print with line numbers)

sed = file.txt | sed 'N;s/\n/\t/'

Combining with Other Tools

# sed + xargs: batch replace in files found by grep
grep -rl 'pattern' . | xargs sed -i 's/pattern/replacement/g'

# sed + awk: sed for line filtering, awk for field extraction
sed -n '/error/p' app.log | awk '{print $1, $NF}'

# sed in a pipeline
cat access.log | sed -n '/POST/p' | cut -d' ' -f7 | sort | uniq -c

Output Format

When responding to a sed request:

1. Show the one-liner in a code block, ready to copy-paste.
2. If editing files in-place, show dry-run first, then the -i version.
3. Add a one-line comment explaining what each part does if non-obvious.
4. If the task is better served by awk, perl, jq, or ripgrep, say so briefly and optionally provide that alternative instead.

Gotchas

• macOS ships BSD sed, not GNU sed. -i requires a mandatory suffix (-i '' is valid on BSD, -i.bak on both). If the user is on macOS, note this.
• Regex in sed is greedy by default; there is no non-greedy quantifier in GNU sed ERE. Workaround: use negated character classes ([^x]* instead of .*?).
• \w is a GNU sed extension, not portable sed. Use [[:alnum:]_] for portable word characters, and [0-9] or [[:digit:]] for digits. Use Perl (perl -pe) when you need PCRE shorthands such as \d.
• Multiline matching across line breaks requires the N command or a hold-space trick. If the user needs real multiline regex, recommend perl -0777 -pe.