Proteome Dataset Handling

proteome-dataset-handling

Summary

Configure and load mass spectrometry proteome datasets (MGF format) into a GPU-accelerated clustering pipeline, matching dataset scale to available GPU memory (GTX 1080Ti for smaller datasets, GTX 3090 for large-scale datasets like PXD000561 with millions of spectra).

When to use

You have a collection of MS/MS spectra in MGF format and need to prepare them for GPU-based clustering. Dataset size and available GPU memory are critical: use GTX 1080Ti for smaller proteome datasets; use GTX 3090 for datasets like PXD000561 that exceed GTX 1080Ti capacity. Preprocessing must normalize spectra by removing precursor peaks, filtering by intensity thresholds, and binning m/z values before encoding into hypervectors.

When NOT to use

• Input spectra are already in hyperdimensional vector format or have been encoded by another method
• Dataset is smaller than a few thousand spectra and does not require GPU acceleration
• Spectra are in formats other than MGF (mzML, mzXML, or other formats must be converted first)

Inputs

• Directory containing MGF files (mass spectrometry peak lists)
• ProteomeXchange dataset identifiers (e.g., PXD000561)
• Preprocessing parameter configuration (peak filtering, intensity scaling, tolerance values)

Outputs

• Preprocessed spectra checkpoint file (HyperSpec checkpoint format)
• Encoded hypervectors in hyperdimensional space (binary representations)
• Dataset metadata: precursor m/z, charge state, retention time per spectrum

How to apply

First, verify your GPU type matches the dataset scale (GTX 3090 for large datasets like PXD000561, GTX 1080Ti for smaller proteomes). Second, organize all MGF files in a single input directory. Third, execute the HyperSpec preprocessing pipeline with appropriate CPU core allocation (default: 6 cores) and filtering parameters: set --min_peaks to exclude low-quality spectra, --min_intensity to filter noise, --max_peaks_used to cap high-intensity peaks, and --remove_precursor_tol (typically 0.5 Da) to eliminate precursor contamination. Fourth, apply intensity scaling (root, log, rank, or off) based on your spectral characteristics. Finally, configure charge filtering (--cluster_charges 2 3) to focus on the most abundant charge states. The preprocessing stage projects spectra into normalized feature space before GPU-accelerated hyperdimensional encoding.

Related tools

• HyperSpec (GPU-accelerated mass spectra clustering framework that loads and preprocesses MGF files into hyperdimensional space) — https://github.com/wh-xu/Hyper-Spec
• Python (Runtime environment for HyperSpec execution; version 3.8+ required)
• CUDA (GPU compute framework enabling GPU-accelerated preprocessing and clustering)

Examples

python src/main.py ~/dataset/ ./output.csv --cpu_core_preprocess=4 --min_peaks=10 --min_intensity=1.0 --max_peaks_used=100 --remove_precursor_tol=0.5 --scaling=root --cluster_charges 2 3

Evaluation signals

• Input MGF directory is accessible and contains valid peak list files with m/z and intensity pairs
• Preprocessing pipeline completes without errors; checkpoint file is created or batch encoding proceeds
• Output spectra metadata includes valid ranges: precursor_mz > 0, precursor_charge ∈ {1,2,3,...}, retention_time ≥ 0
• Number of spectra after filtering matches expected retention (e.g., after charge filtering and peak count thresholds)
• GPU memory utilization remains within limits for the selected GPU type (no out-of-memory errors during encoding)

Limitations

• Clustering for large datasets like PXD000561 requires GTX 3090 GPU with larger memory; GTX 1080Ti will fail on such datasets
• System requires Linux platform with properly configured CUDA environment; Windows and macOS support not documented
• HyperSpec has been tested only on GTX 1080Ti and GTX 3090; other NVIDIA GPUs may require additional validation
• High-performance SSD storage is recommended for best performance; standard storage may create I/O bottlenecks with large datasets
• MGF format is required; other mass spectrometry formats (mzML, mzXML) must be converted beforehand

Evidence

• [readme] System Requirements: HyperSpec requires Python 3.8+ with CUDA environment: "HyperSpec requires Python 3.8+ with CUDA environment. A GPU should be installed properly."
• [readme] GPU allocation by dataset scale: "Clustering for PXD000561 dataset requires GTX 3090 with larger memory. Clustering for other dataset with smaller scale requires GTX 1080Ti."
• [readme] Input format and preprocessing workflow: "HyperSpec supports running using the command line and takes MGF peak files as input and exports the clustering result as a CSV file"
• [readme] Preprocessing parameters for dataset handling: "--cpu_core_preprocess [CPU_CORE_PREPROCESS], --min_peaks [MIN_PEAKS], --min_intensity [MIN_INTENSITY], --max_peaks_used [MAX_PEAKS_USED], --scaling {off,root,log,rank}, --remove_precursor_tol"
• [readme] Performance-critical infrastructure recommendation: "We recommend using high-performance SSD as the storage device for the best performance."
• [readme] Brain-inspired hyperdimensional encoding and bucketing: "HyperSpec first encodes the processed spectra into binary hypervector (HV) with ultra-high dimension (>1000) based on level-id encoding method."