pavel-kravchenko Regular Expressions for Bioinformatics
When to Use
- Finding ORFs (ATG...stop codon) or start/stop codon positions in a DNA sequence
- Locating restriction enzyme cut sites, including degenerate (IUPAC-ambiguous) recognition sequences
- Detecting homopolymer runs or tandem repeats (microsatellites)
- Parsing FASTA/GenBank/UniProt headers or BLAST tabular output into structured fields
- Matching degenerate PCR primers or PROSITE-style protein motifs
Version Compatibility
Python ≥3.8 stdlib re module — no external dependencies, stable behavior across 3.8–3.13. For variable-length lookbehind or fuzzy matching, the third-party regex package (PyPI) is a drop-in extension; for large PROSITE/Pfam-scale motif scans, Bio.motifs (Biopython) is more appropriate than hand-rolled regex.
Prerequisites
- Python string basics, DNA/RNA/protein alphabets
- Familiarity with FASTA/GenBank text layout
- Related skills:
python-bio-strings,bio-sequence-io-read-sequences,bio-restriction-analysis-restriction-sites
Quick Reference
| Function | Returns | Use |
|---|---|---|
re.search(pat, s) |
First match object or None | Check if pattern exists |
re.findall(pat, s) |
List of strings (or tuples with groups) | All non-overlapping matches |
re.finditer(pat, s) |
Iterator of match objects | All matches with positions |
re.sub(pat, repl, s) |
Modified string | Replace matches |
re.split(pat, s) |
List of strings | Split on pattern |
re.compile(pat) |
Compiled regex | Reuse expensive patterns |
Lookaround Quick Reference
| Syntax | Meaning | Bio use |
|---|---|---|
(?=pat) |
Positive lookahead | Overlapping matches; site not consuming |
(?!pat) |
Negative lookahead | ATG not followed by a stop codon |
(?<=pat) |
Positive lookbehind | Bases after a restriction site |
(?<!pat) |
Negative lookbehind | Context-excluded matches |
Goal: Find all open reading frames (ATG ... stop codon) in all three forward reading frames. Approach: compile a pattern with a non-greedy repeated-codon body so it stops at the nearest in-frame stop, then re-scan the sequence shifted by 0/1/2 bases for each frame.
import re
ORF_PATTERN = re.compile(r'ATG(?:[ATGC]{3})*?(?:TAA|TAG|TGA)')
def find_orfs(sequence, min_length=30):
"""Find ORFs (ATG...stop) in all 3 forward reading frames.
Returns a list of dicts (frame, start, end, length, sequence),
sorted longest first. Coordinates are 0-based.
"""
orfs = []
for frame in range(3):
for m in ORF_PATTERN.finditer(sequence[frame:]):
orf_seq = m.group()
if len(orf_seq) >= min_length:
orfs.append({
'frame': frame + 1,
'start': m.start() + frame,
'end': m.end() + frame,
'length': len(orf_seq),
'sequence': orf_seq,
})
return sorted(orfs, key=lambda o: o['length'], reverse=True)
dna = "ATGAAAGCCTTTGGGATCGATCGTAGATGCCCCCCGGGATCGATCGATCGTGA"
for orf in find_orfs(dna, min_length=9):
print(orf['frame'], orf['start'], orf['end'], orf['length'])
Goal: Map restriction-enzyme cut sites, including degenerate (IUPAC) recognition sequences and tandem repeats.
Approach: translate IUPAC ambiguity codes to regex character classes before matching (IUPAC N is not a regex wildcard), then use a zero-width lookahead in finditer so overlapping sites aren't missed.
import re
IUPAC = {'A': 'A', 'T': 'T', 'G': 'G', 'C': 'C', 'N': '[ATGC]', 'R': '[AG]',
'Y': '[CT]', 'W': '[AT]', 'S': '[GC]', 'M': '[AC]', 'K': '[GT]',
'B': '[CGT]', 'D': '[AGT]', 'H': '[ACT]', 'V': '[ACG]'}
def iupac_to_regex(seq):
"""Translate an IUPAC-ambiguous sequence (e.g. a degenerate primer) to a regex."""
return ''.join(IUPAC.get(b, b) for b in seq.upper())
def find_cut_sites(dna, site, cut_offset=0):
"""Find all (overlapping) cut positions for a recognition site.
site: IUPAC sequence, e.g. 'GAATTC' for EcoRI.
cut_offset: bases into the site where the enzyme cuts
(EcoRI cuts G^AATTC -> cut_offset=1).
"""
pattern = iupac_to_regex(site)
return [m.start() + cut_offset for m in re.finditer(f'(?={pattern})', dna.upper())]
def find_tandem_repeats(sequence, unit_len=3, min_copies=3):
"""Find tandem repeats of a given unit length, e.g. microsatellites."""
pattern = re.compile(r'(([ATGC]{%d})\2{%d,})' % (unit_len, min_copies - 1))
return [(m.start(), m.group(2), len(m.group(1)) // unit_len) for m in pattern.finditer(sequence)]
dna = "ATGCGAATTCGATCGATCGAATTCGATCG"
print(find_cut_sites(dna, 'GAATTC', cut_offset=1)) # EcoRI: G^AATTC
print(find_tandem_repeats("ATGCACACACACACACATGC", unit_len=2, min_copies=3))
Goal: Parse UniProt/GenBank-style FASTA headers and convert PROSITE motif notation to regex.
Approach: use named capture groups for headers so fields are pulled out by name, not position; walk the PROSITE string character-by-character translating [..], {..} (negated class), x (any residue), and (n,m) (repeat count) into standard regex syntax.
import re
def parse_uniprot_header(header):
"""Parse a UniProt FASTA header, e.g. '>sp|P04637|P53_HUMAN ... OS=Homo sapiens'."""
pattern = (r'>sp\|(?P<accession>[^|]+)\|(?P<entry_name>\S+)\s+'
r'(?P<description>.+?)\s+OS=(?P<organism>.+)')
m = re.search(pattern, header)
return m.groupdict() if m else None
def prosite_to_regex(pattern):
"""Convert a PROSITE motif (e.g. 'N-{P}-[ST]-{P}') to a Python regex.
[ABC] -> matches A, B, or C; {ABC} -> matches anything except A/B/C;
x -> any residue; (n) or (n,m) -> repeat count; '-' is a separator.
"""
result = []
parts = pattern.replace('-', '')
i = 0
while i < len(parts):
if parts[i] == '[':
end = parts.index(']', i)
result.append(parts[i:end + 1])
i = end + 1
elif parts[i] == '{':
end = parts.index('}', i)
result.append(f'[^{parts[i + 1:end]}]')
i = end + 1
elif parts[i] == 'x':
result.append('.')
i += 1
elif parts[i] == '(':
end = parts.index(')', i)
result.append('{' + parts[i + 1:end] + '}')
i = end + 1
else:
result.append(parts[i])
i += 1
return ''.join(result)
header = ">sp|P04637|P53_HUMAN Cellular tumor antigen p53 OS=Homo sapiens"
print(parse_uniprot_header(header))
glyco = re.compile(prosite_to_regex('N-{P}-[ST]-{P}')) # N-glycosylation motif
protein = "MNGTSAKNLTCVNSTGMKNQTAPRSNVTKGAISSMSDELLNK"
print([(m.start() + 1, m.group()) for m in glyco.finditer(protein)])
Pitfalls
- Greedy vs non-greedy:
ATG.*TAGmatches ATG to the last TAG in the string;ATG.*?TAGstops at the nearest — always choose deliberately for ORF searches - Overlapping matches:
re.findall('ATG', dna)skips overlaps; usere.finditer(r'(?=(ATG))', dna)to find all - IUPAC codes are not regex:
Nin a sequence means any base, butNin a regex matches the literalN— always translate withiupac_to_regex()first re.MULTILINEfor FASTA:^>only anchors to the start of the whole string by default; addre.MULTILINEto match each linefindallwith groups: with one capturing group returns group contents (not full match); with multiple groups returns a list of tuples- Off-by-one with
match.start(): positions are 0-based; bioinformatics coordinates are often 1-based — add 1 when reporting
See Also
python-bio-strings— string manipulation fundamentals used alongside regexbio-sequence-io-read-sequences— reading FASTA/FASTQ before/after regex parsingbio-restriction-analysis-restriction-sites— dedicated restriction-mapping workflowsbio-sequence-manipulation-motif-search— motif search beyond regex (PWMs, profiles)