Bioinformatics and Genomics File Formats Reference

Bioinformatics and Genomics File Formats Reference

This reference covers file formats used in genomics, transcriptomics, sequence analysis, and related bioinformatics applications.

Sequence Data Formats

.fasta / .fa / .fna - FASTA Format

Description: Text-based format for nucleotide or protein sequences Typical Data: DNA, RNA, or protein sequences with headers Use Cases: Sequence storage, BLAST searches, alignments Python Libraries:

• Biopython: SeqIO.parse('file.fasta', 'fasta')
• pyfaidx: Fast indexed FASTA access
• screed: Fast sequence parsing EDA Approach:
• Sequence count and length distribution
• GC content analysis
• N content (ambiguous bases)
• Sequence ID parsing
• Duplicate detection
• Quality metrics for assemblies (N50, L50)

.fastq / .fq - FASTQ Format

Description: Sequence data with base quality scores Typical Data: Raw sequencing reads with Phred quality scores Use Cases: NGS data, quality control, read mapping Python Libraries:

• Biopython: SeqIO.parse('file.fastq', 'fastq')
• pysam: Fast FASTQ/BAM operations
• HTSeq: Sequencing data analysis EDA Approach:
• Read count and length distribution
• Quality score distribution (per-base, per-read)
• GC content and bias
• Duplicate rate estimation
• Adapter contamination detection
• k-mer frequency analysis
• Encoding format validation (Phred33/64)

.sam - Sequence Alignment/Map

Description: Tab-delimited text format for alignments Typical Data: Aligned sequencing reads with mapping quality Use Cases: Read alignment storage, variant calling Python Libraries:

• pysam: pysam.AlignmentFile('file.sam', 'r')
• HTSeq: HTSeq.SAM_Reader('file.sam') EDA Approach:
• Mapping rate and quality distribution
• Coverage analysis
• Insert size distribution (paired-end)
• Alignment flags distribution
• CIGAR string patterns
• Mismatch and indel rates
• Duplicate and supplementary alignment counts

.bam - Binary Alignment/Map

Description: Compressed binary version of SAM Typical Data: Aligned reads in compressed format Use Cases: Efficient storage and processing of alignments Python Libraries:

• pysam: Full BAM support with indexing
• bamnostic: Pure Python BAM reader EDA Approach:
• Same as SAM plus:
• Compression ratio analysis
• Index file (.bai) validation
• Chromosome-wise statistics
• Strand bias detection
• Read group analysis

.cram - CRAM Format

Description: Highly compressed alignment format Typical Data: Reference-compressed aligned reads Use Cases: Long-term storage, space-efficient archives Python Libraries:

• pysam: CRAM support (requires reference)
• Reference genome must be accessible EDA Approach:
• Compression efficiency vs BAM
• Reference dependency validation
• Lossy vs lossless compression assessment
• Decompression performance
• Similar alignment metrics as BAM

.bed - Browser Extensible Data

Description: Tab-delimited format for genomic features Typical Data: Genomic intervals (chr, start, end) with annotations Use Cases: Peak calling, variant annotation, genome browsing Python Libraries:

• pybedtools: pybedtools.BedTool('file.bed')
• pyranges: pyranges.read_bed('file.bed')
• pandas: Simple BED reading EDA Approach:
• Feature count and size distribution
• Chromosome distribution
• Strand bias
• Score distribution (if present)
• Overlap and proximity analysis
• Coverage statistics
• Gap analysis between features

.bedGraph - BED with Graph Data

Description: BED format with per-base signal values Typical Data: Continuous-valued genomic data (coverage, signals) Use Cases: Coverage tracks, ChIP-seq signals, methylation Python Libraries:

• pyBigWig: Can convert to bigWig
• pybedtools: BedGraph operations EDA Approach:
• Signal distribution statistics
• Genome coverage percentage
• Signal dynamics (peaks, valleys)
• Chromosome-wise signal patterns
• Quantile analysis
• Zero-coverage regions

.bigWig / .bw - Binary BigWig

Description: Indexed binary format for genome-wide signal data Typical Data: Continuous genomic signals (compressed and indexed) Use Cases: Efficient genome browser tracks, large-scale data Python Libraries:

• pyBigWig: pyBigWig.open('file.bw')
• pybbi: BigWig/BigBed interface EDA Approach:
• Signal statistics extraction
• Zoom level analysis
• Regional signal extraction
• Efficient genome-wide summaries
• Compression efficiency
• Index structure analysis

.bigBed / .bb - Binary BigBed

Description: Indexed binary BED format Typical Data: Genomic features (compressed and indexed) Use Cases: Large feature sets, genome browsers Python Libraries:

• pybbi: BigBed reading
• pybigtools: Modern BigBed interface EDA Approach:
• Feature density analysis
• Efficient interval queries
• Zoom level validation
• Index performance metrics
• Feature size statistics

.gff / .gff3 - General Feature Format

Description: Tab-delimited format for genomic annotations Typical Data: Gene models, transcripts, exons, regulatory elements Use Cases: Genome annotation, gene prediction Python Libraries:

• BCBio.GFF: Biopython GFF module
• gffutils: gffutils.create_db('file.gff3')
• pyranges: GFF support EDA Approach:
• Feature type distribution (gene, exon, CDS, etc.)
• Gene structure validation
• Strand balance
• Hierarchical relationship validation
• Phase validation for CDS
• Attribute completeness
• Gene model statistics (introns, exons per gene)

.gtf - Gene Transfer Format

Description: GFF2-based format for gene annotations Typical Data: Gene and transcript annotations Use Cases: RNA-seq analysis, gene quantification Python Libraries:

• pyranges: pyranges.read_gtf('file.gtf')
• gffutils: GTF database creation
• HTSeq: GTF reading for counts EDA Approach:
• Transcript isoform analysis
• Gene structure completeness
• Exon number distribution
• Transcript length distribution
• TSS and TES analysis
• Biotype distribution
• Overlapping gene detection

.vcf - Variant Call Format

Description: Text format for genetic variants Typical Data: SNPs, indels, structural variants with annotations Use Cases: Variant calling, population genetics, GWAS Python Libraries:

• pysam: pysam.VariantFile('file.vcf')
• cyvcf2: Fast VCF parsing
• PyVCF: Older but comprehensive EDA Approach:
• Variant count by type (SNP, indel, SV)
• Quality score distribution
• Allele frequency spectrum
• Transition/transversion ratio
• Heterozygosity rates
• Missing genotype analysis
• Hardy-Weinberg equilibrium
• Annotation completeness (if annotated)

.bcf - Binary VCF

Description: Compressed binary variant format Typical Data: Same as VCF but binary Use Cases: Efficient variant storage and processing Python Libraries:

• pysam: Full BCF support
• cyvcf2: Optimized BCF reading EDA Approach:
• Same as VCF plus:
• Compression efficiency
• Indexing validation
• Read performance metrics

.gvcf - Genomic VCF

Description: VCF with reference confidence blocks Typical Data: All positions (variant and non-variant) Use Cases: Joint genotyping workflows, GATK Python Libraries:

• pysam: GVCF support
• Standard VCF parsers EDA Approach:
• Reference block analysis
• Coverage uniformity
• Variant density
• Genotype quality across genome
• Reference confidence distribution

RNA-Seq and Expression Data

.counts - Gene Count Matrix

Description: Tab-delimited gene expression counts Typical Data: Gene IDs with read counts per sample Use Cases: RNA-seq quantification, differential expression Python Libraries:

• pandas: pd.read_csv('file.counts', sep='\t')
• scanpy (for single-cell): sc.read_csv() EDA Approach:
• Library size distribution
• Detection rate (genes per sample)
• Zero-inflation analysis
• Count distribution (log scale)
• Outlier sample detection
• Correlation between replicates
• PCA for sample relationships

.tpm / .fpkm - Normalized Expression

Description: Normalized gene expression values Typical Data: TPM (transcripts per million) or FPKM values Use Cases: Cross-sample comparison, visualization Python Libraries:

• pandas: Standard CSV reading
• anndata: For integrated analysis EDA Approach:
• Expression distribution
• Highly expressed gene identification
• Sample clustering
• Batch effect detection
• Coefficient of variation analysis
• Dynamic range assessment

.mtx - Matrix Market Format

Description: Sparse matrix format (common in single-cell) Typical Data: Sparse count matrices (cells × genes) Use Cases: Single-cell RNA-seq, large sparse matrices Python Libraries:

• scipy.io: scipy.io.mmread('file.mtx')
• scanpy: sc.read_mtx('file.mtx') EDA Approach:
• Sparsity analysis
• Cell and gene filtering thresholds
• Doublet detection metrics
• Mitochondrial fraction
• UMI count distribution
• Gene detection per cell

.h5ad - Anndata Format

Description: HDF5-based annotated data matrix Typical Data: Expression matrix with metadata (cells, genes) Use Cases: Single-cell RNA-seq analysis with Scanpy Python Libraries:

• scanpy: sc.read_h5ad('file.h5ad')
• anndata: Direct AnnData manipulation EDA Approach:
• Cell and gene counts
• Metadata completeness
• Layer availability (raw, normalized)
• Embedding presence (PCA, UMAP)
• QC metrics distribution
• Batch information
• Cell type annotation coverage

.loom - Loom Format

Description: HDF5-based format for omics data Typical Data: Expression matrices with metadata Use Cases: Single-cell data, RNA velocity analysis Python Libraries:

• loompy: loompy.connect('file.loom')
• scanpy: Can import loom files EDA Approach:
• Layer analysis (spliced, unspliced)
• Row and column attribute exploration
• Graph connectivity analysis
• Cluster assignments
• Velocity-specific metrics

.rds - R Data Serialization

Description: R object storage (often Seurat objects) Typical Data: R analysis results, especially single-cell Use Cases: R-Python data exchange Python Libraries:

• pyreadr: pyreadr.read_r('file.rds')
• rpy2: For full R integration
• Conversion tools to AnnData EDA Approach:
• Object type identification
• Data structure exploration
• Metadata extraction
• Conversion validation

Alignment and Assembly Formats

.maf - Multiple Alignment Format

Description: Text format for multiple sequence alignments Typical Data: Genome-wide or local multiple alignments Use Cases: Comparative genomics, conservation analysis Python Libraries:

• Biopython: AlignIO.parse('file.maf', 'maf')
• bx-python: MAF-specific tools EDA Approach:
• Alignment block statistics
• Species coverage
• Gap analysis
• Conservation scoring
• Alignment quality metrics
• Block length distribution

.axt - Pairwise Alignment Format

Description: Pairwise alignment format (UCSC) Typical Data: Pairwise genomic alignments Use Cases: Genome comparison, synteny analysis Python Libraries:

• Custom parsers (simple format)
• bx-python: AXT support EDA Approach:
• Alignment score distribution
• Identity percentage
• Syntenic block identification
• Gap size analysis
• Coverage statistics

.chain - Chain Alignment Format

Description: Genome coordinate mapping chains Typical Data: Coordinate transformations between genome builds Use Cases: Liftover, coordinate conversion Python Libraries:

• pyliftover: Chain file usage
• Custom parsers for chain format EDA Approach:
• Chain score distribution
• Coverage of source genome
• Gap analysis
• Inversion detection
• Mapping quality assessment

.psl - Pattern Space Layout

Description: BLAT/BLAST alignment format Typical Data: Alignment results from BLAT Use Cases: Transcript mapping, similarity searches Python Libraries:

• Custom parsers (tab-delimited)
• pybedtools: Can handle PSL EDA Approach:
• Match percentage distribution
• Gap statistics
• Query coverage
• Multiple mapping analysis
• Alignment quality metrics

Genome Assembly and Annotation

.agp - Assembly Golden Path

Description: Assembly structure description Typical Data: Scaffold composition, gap information Use Cases: Genome assembly representation Python Libraries:

• Custom parsers (simple tab-delimited)
• Assembly analysis tools EDA Approach:
• Scaffold statistics (N50, L50)
• Gap type and size distribution
• Component length analysis
• Assembly contiguity metrics
• Unplaced contig analysis

.scaffolds / .contigs - Assembly Sequences

Description: Assembled sequences (usually FASTA) Typical Data: Assembled genomic sequences Use Cases: Genome assembly output Python Libraries:

• Same as FASTA format
• Assembly-specific tools (QUAST) EDA Approach:
• Assembly statistics (N50, N90, etc.)
• Length distribution
• Coverage analysis
• Gap (N) content
• Duplication assessment
• BUSCO completeness (if annotations available)

.2bit - Compressed Genome Format

Description: UCSC compact genome format Typical Data: Reference genomes (highly compressed) Use Cases: Efficient genome storage and access Python Libraries:

• py2bit: py2bit.open('file.2bit')
• twobitreader: Alternative reader EDA Approach:
• Compression efficiency
• Random access performance
• Sequence extraction validation
• Masked region analysis
• N content and distribution

.sizes - Chromosome Sizes

Description: Simple format with chromosome lengths Typical Data: Tab-delimited chromosome names and sizes Use Cases: Genome browsers, coordinate validation Python Libraries:

• Simple file reading with pandas
• Built into many genomic tools EDA Approach:
• Genome size calculation
• Chromosome count
• Size distribution
• Karyotype validation
• Completeness check against reference

Phylogenetics and Evolution

.nwk / .newick - Newick Tree Format

Description: Parenthetical tree representation Typical Data: Phylogenetic trees with branch lengths Use Cases: Evolutionary analysis, tree visualization Python Libraries:

• Biopython: Phylo.read('file.nwk', 'newick')
• ete3: ete3.Tree('file.nwk')
• dendropy: Phylogenetic computing EDA Approach:
• Tree structure analysis (tips, internal nodes)
• Branch length distribution
• Tree balance metrics
• Ultrametricity check
• Bootstrap support analysis
• Topology validation

.nexus - Nexus Format

Description: Rich format for phylogenetic data Typical Data: Alignments, trees, character matrices Use Cases: Phylogenetic software interchange Python Libraries:

• Biopython: Nexus support
• dendropy: Comprehensive Nexus handling EDA Approach:
• Data block analysis
• Character type distribution
• Tree block validation
• Taxa consistency
• Command block parsing
• Format compliance checking

.phylip - PHYLIP Format

Description: Sequence alignment format (strict/relaxed) Typical Data: Multiple sequence alignments Use Cases: Phylogenetic analysis input Python Libraries:

• Biopython: AlignIO.read('file.phy', 'phylip')
• dendropy: PHYLIP support EDA Approach:
• Alignment dimensions
• Sequence length uniformity
• Gap position analysis
• Informative site calculation
• Format variant detection (strict vs relaxed)

.paml - PAML Output

Description: Output from PAML phylogenetic software Typical Data: Evolutionary model results, dN/dS ratios Use Cases: Molecular evolution analysis Python Libraries:

• Custom parsers for specific PAML programs
• Biopython: Basic PAML parsing EDA Approach:
• Model parameter extraction
• Likelihood values
• dN/dS ratio distribution
• Branch-specific results
• Convergence assessment

Protein and Structure Data

.embl - EMBL Format

Description: Rich sequence annotation format Typical Data: Sequences with extensive annotations Use Cases: Sequence databases, genome records Python Libraries:

• Biopython: SeqIO.read('file.embl', 'embl') EDA Approach:
• Feature annotation completeness
• Sequence length and type
• Reference information
• Cross-reference validation
• Feature overlap analysis

.genbank / .gb / .gbk - GenBank Format

Description: NCBI's sequence annotation format Typical Data: Annotated sequences with features Use Cases: Sequence databases, annotation transfer Python Libraries:

• Biopython: SeqIO.parse('file.gb', 'genbank') EDA Approach:
• Feature type distribution
• CDS analysis (start codons, stops)
• Translation validation
• Annotation completeness
• Source organism extraction
• Reference and publication info
• Locus tag consistency

.sff - Standard Flowgram Format

Description: 454/Roche sequencing data format Typical Data: Raw pyrosequencing flowgrams Use Cases: Legacy 454 sequencing data Python Libraries:

• Biopython: SeqIO.parse('file.sff', 'sff')
• Platform-specific tools EDA Approach:
• Read count and length
• Flowgram signal quality
• Key sequence detection
• Adapter trimming validation
• Quality score distribution

.hdf5 (Genomics Specific)

Description: HDF5 for genomics (10X, Hi-C, etc.) Typical Data: High-throughput genomics data Use Cases: 10X Genomics, spatial transcriptomics Python Libraries:

• h5py: Low-level access
• scanpy: For 10X data
• cooler: For Hi-C data EDA Approach:
• Dataset structure exploration
• Barcode statistics
• UMI counting
• Feature-barcode matrix analysis
• Spatial coordinates (if applicable)

.cool / .mcool - Cooler Format

Description: HDF5-based Hi-C contact matrices Typical Data: Chromatin interaction matrices Use Cases: 3D genome analysis, Hi-C data Python Libraries:

• cooler: cooler.Cooler('file.cool')
• hicstraw: For .hic format EDA Approach:
• Resolution analysis
• Contact matrix statistics
• Distance decay curves
• Compartment analysis
• TAD boundary detection
• Balance factor validation

.hic - Hi-C Binary Format

Description: Juicer binary Hi-C format Typical Data: Multi-resolution Hi-C matrices Use Cases: Hi-C analysis with Juicer tools Python Libraries:

• hicstraw: hicstraw.HiCFile('file.hic')
• straw: C++ library with Python bindings EDA Approach:
• Available resolutions
• Normalization methods
• Contact statistics
• Chromosomal interactions
• Quality metrics

.bw (ChIP-seq / ATAC-seq specific)

Description: BigWig files for epigenomics Typical Data: Coverage or enrichment signals Use Cases: ChIP-seq, ATAC-seq, DNase-seq Python Libraries:

• pyBigWig: Standard bigWig access EDA Approach:
• Peak enrichment patterns
• Background signal analysis
• Sample correlation
• Signal-to-noise ratio
• Library complexity metrics

.narrowPeak / .broadPeak - ENCODE Peak Formats

Description: BED-based formats for peaks Typical Data: Peak calls with scores and p-values Use Cases: ChIP-seq peak calling output Python Libraries:

• pybedtools: BED-compatible
• Custom parsers for peak-specific fields EDA Approach:
• Peak count and width distribution
• Signal value distribution
• Q-value and p-value analysis
• Peak summit analysis
• Overlap with known features
• Motif enrichment preparation

.wig - Wiggle Format

Description: Dense continuous genomic data Typical Data: Coverage or signal tracks Use Cases: Genome browser visualization Python Libraries:

• pyBigWig: Can convert to bigWig
• Custom parsers for wiggle format EDA Approach:
• Signal statistics
• Coverage metrics
• Format variant (fixedStep vs variableStep)
• Span parameter analysis
• Conversion efficiency to bigWig

.ab1 - Sanger Sequencing Trace

Description: Binary chromatogram format Typical Data: Sanger sequencing traces Use Cases: Capillary sequencing validation Python Libraries:

• Biopython: SeqIO.read('file.ab1', 'abi')
• tracy tools: For quality assessment EDA Approach:
• Base calling quality
• Trace quality scores
• Mixed base detection
• Primer and vector detection
• Read length and quality region
• Heterozygosity detection

.scf - Standard Chromatogram Format

Description: Sanger sequencing chromatogram Typical Data: Base calls and confidence values Use Cases: Sequencing trace analysis Python Libraries:

• Biopython: SCF format support EDA Approach:
• Similar to AB1 format
• Quality score profiles
• Peak height ratios
• Signal-to-noise metrics

.idx - Index Files (Generic)

Description: Index files for various formats Typical Data: Fast random access indices Use Cases: Efficient data access (BAM, VCF, etc.) Python Libraries:

• Format-specific libraries handle indices
• pysam: Auto-handles BAI, CSI indices EDA Approach:
• Index completeness validation
• Binning strategy analysis
• Access performance metrics
• Index size vs data size ratio