sample-type-stratified-feature-masking
Summary
Remove features from a metabolomics feature table whose intensity in study (unknown) samples is not sufficiently elevated above blank samples, using a configurable intensity ratio threshold. This quality-control step eliminates likely contaminants and noise that appear equally in blanks and unknowns.
When to use
Apply this skill after feature detection but before statistical analysis, when your study includes blank samples (e.g., solvent or extraction blanks) and you want to remove features that fail to show meaningful enrichment in actual study samples relative to blank contamination. Typical trigger: presence of a 'sample_type' or analogous metadata field that designates some samples as blanks and others as unknowns or QC.
When NOT to use
- Input study lacks blank samples or blank/unknown sample stratification is unavailable.
- Feature table is already post-QC and blanks have been removed; blank-masking requires the blank samples to be present in the input table.
- Blanks are QC or standard mixtures rather than true process blanks—intensity may legitimately exceed study samples.
Inputs
- feature table (intensity matrix with features as rows, samples as columns)
- sample metadata / sequence file with sample_type or equivalent classification field
Outputs
- filtered feature table (subset of input, retaining only features with unknown/blank intensity ratio ≥ threshold)
- optionally: masking report or QC visualization showing features retained vs. removed
How to apply
Load the feature table and sample metadata, stratify samples into blank and unknown groups using a metadata query field (e.g., sample_type='blank' vs. sample_type='unknown'). For each feature, calculate median or mean intensity across blanks and across unknowns, excluding zero values to avoid inflation from missing features. Compute the intensity ratio (unknown / blank) for each feature and retain only features where this ratio meets or exceeds the blank_intensity_ratio threshold (default 3, meaning unknown intensity must be ≥ 3× blank intensity). Save the filtered table with a new moniker, preserving all metadata and feature annotations for downstream analysis.
Related tools
Evaluation signals
- Feature count decreases; verify that removed features have unknown/blank intensity ratios below the specified threshold (e.g., < 3).
- Retained features show elevated median intensity in unknown samples relative to blanks (ratio typically > 3); spot-check a sample of removed features to confirm they were contamination-like or noise.
- Sample metadata and feature annotations (m/z, retention time, etc.) are preserved in output table; verify schema and row/column structure match input.
- Output table row count equals input minus removed features; output column count (samples) is unchanged.
- Optionally compare PCA, t-SNE, or other exploratory plots before and after masking—blank samples should cluster separately from unknowns pre-masking; post-masking, unknowns should show clearer separation or reduced noise.
Limitations
- Effectiveness depends on quality and representativeness of blank samples; if blanks do not reflect true process contamination, intensity ratio may be misleading.
- Default threshold (ratio = 3) is data-dependent and may not suit all metabolite classes or instrumental platforms; users should validate via domain knowledge or pilot experiments.
- Method assumes intensity scales linearly with contamination; non-linear instrumental responses or matrix effects could bias the ratio.
- Exclusion of zero values before computing mean/median avoids division-by-zero but may artificially inflate ratios if blanks contain many missing values and unknowns contain sparse detections.
- README notes that support for GC and other data types is under development; currently documented for LC-MS only.
Evidence
- [other] The blank_masking command removes features by comparing feature intensity in study samples (designated by sample_value) to intensity in blanks (designated by blank_value) using a configurable blank_intensity_ratio parameter; features whose intensity in unknown samples is not at least the specified ratio times more than blank samples are dropped.: "blank_masking command removes features by comparing feature intensity in study samples (designated by sample_value) to intensity in blanks (designated by blank_value) using a configurable"
- [other] For each feature, calculate the median or mean intensity across blank samples and across unknown samples, excluding zero values. Compute the intensity ratio (unknown/blank) for each feature and compare against the blank_intensity_ratio threshold (default 3). Retain only features where unknown sample intensity exceeds blank intensity by at least the threshold ratio.: "For each feature, calculate the median or mean intensity across blank samples and across unknown samples, excluding zero values. Compute the intensity ratio (unknown/blank) for each feature and"
- [other] Load the input feature table and sample metadata using the specified table moniker and identify blank and unknown samples based on the query field (e.g., 'sample_type').: "Load the input feature table and sample metadata using the specified table moniker and identify blank and unknown samples based on the query field (e.g., 'sample_type')"
- [other] Save the filtered feature table with a new moniker, preserving metadata and feature annotations.: "Save the filtered feature table with a new moniker, preserving metadata and feature annotations"
- [readme] The Python-Centric Pipeline for Metabolomics is designed to take raw LC-MS metabolomics data and ready them for downstream statistical analysis. The pipeline can perform quality control, data normalization and batch correction.: "perform quality control, data normalization and batch correction"
1---2name: sample-type-stratified-feature-masking-33description: Use when after feature detection but before statistical analysis, when4 your study includes blank samples (e.g., solvent or extraction blanks) and you want5 to remove features that fail to show meaningful enrichment in actual study samples6 relative to blank contamination.7license: CC-BY-4.08metadata:9 source: https://skillmd.com/skills/holobiomicslab/sample-type-stratified-feature-masking-310 edam_operation: http://edamontology.org/operation_369511 edam_topics:12 - http://edamontology.org/topic_009113 - http://edamontology.org/topic_337514 tools:15 - ThermoRawFileParser16 - Python17 - PCPFM (Python-Centric Pipeline for Metabolomics)18 - Asari19 techniques:20 - LC-MS21 license_tier: open22 provenance_tier: literature23derived_from:24- doi: 10.1371/journal.pcbi.101191225 title: pcpfm26evidence_spans:27- convert Thermo .raw to mzML (ThermoRawFileParser)28- Python-Centric Pipeline for Metabolomics29- The Python-Centric Pipeline for Metabolomics30claims: []31provenance:32 collection: https://w3id.org/holobiomicslab/asb-skill/collection/metabolomics/v233 assembled_by: scripts/collect_metabolomics_collection.py34 sources:35 - build: coll_pcpfm36 doi: 10.1371/journal.pcbi.101191237 title: pcpfm38 dedup_kept_from: coll_pcpfm39schema_version: 0.2.040attribution:41 generator: AgenticScienceBuilder42 original_doi: 10.1371/journal.pcbi.101191243 all_source_dois:44 - 10.1371/journal.pcbi.101191245 zenodo_doi: 10.5281/zenodo.2079402746 curators: []47 promoter: Louis-Félix Nothias48 sponsor: CNRS & Université Côte d'Azur49---5051# sample-type-stratified-feature-masking5253## Summary5455Remove features from a metabolomics feature table whose intensity in study (unknown) samples is not sufficiently elevated above blank samples, using a configurable intensity ratio threshold. This quality-control step eliminates likely contaminants and noise that appear equally in blanks and unknowns.5657## When to use5859Apply this skill after feature detection but before statistical analysis, when your study includes blank samples (e.g., solvent or extraction blanks) and you want to remove features that fail to show meaningful enrichment in actual study samples relative to blank contamination. Typical trigger: presence of a 'sample_type' or analogous metadata field that designates some samples as blanks and others as unknowns or QC.6061## When NOT to use6263- Input study lacks blank samples or blank/unknown sample stratification is unavailable.64- Feature table is already post-QC and blanks have been removed; blank-masking requires the blank samples to be present in the input table.65- Blanks are QC or standard mixtures rather than true process blanks—intensity may legitimately exceed study samples.6667## Inputs6869- feature table (intensity matrix with features as rows, samples as columns)70- sample metadata / sequence file with sample_type or equivalent classification field7172## Outputs7374- filtered feature table (subset of input, retaining only features with unknown/blank intensity ratio ≥ threshold)75- optionally: masking report or QC visualization showing features retained vs. removed7677## How to apply7879Load the feature table and sample metadata, stratify samples into blank and unknown groups using a metadata query field (e.g., sample_type='blank' vs. sample_type='unknown'). For each feature, calculate median or mean intensity across blanks and across unknowns, excluding zero values to avoid inflation from missing features. Compute the intensity ratio (unknown / blank) for each feature and retain only features where this ratio meets or exceeds the blank_intensity_ratio threshold (default 3, meaning unknown intensity must be ≥ 3× blank intensity). Save the filtered table with a new moniker, preserving all metadata and feature annotations for downstream analysis.8081## Related tools8283- **PCPFM (Python-Centric Pipeline for Metabolomics)** (orchestrates blank_masking as a configurable QC step within the end-to-end LC-MS preprocessing workflow) — https://github.com/shuzhao-li-lab/PythonCentricPipelineForMetabolomics84- **Python** (implementation language for blank_masking command and intensity ratio calculations)85- **Asari** (produces the initial feature table (via mzML processing) that serves as input to blank masking) — https://github.com/shuzhao-li/asari8687## Evaluation signals8889- Feature count decreases; verify that removed features have unknown/blank intensity ratios below the specified threshold (e.g., < 3).90- Retained features show elevated median intensity in unknown samples relative to blanks (ratio typically > 3); spot-check a sample of removed features to confirm they were contamination-like or noise.91- Sample metadata and feature annotations (m/z, retention time, etc.) are preserved in output table; verify schema and row/column structure match input.92- Output table row count equals input minus removed features; output column count (samples) is unchanged.93- Optionally compare PCA, t-SNE, or other exploratory plots before and after masking—blank samples should cluster separately from unknowns pre-masking; post-masking, unknowns should show clearer separation or reduced noise.9495## Limitations9697- Effectiveness depends on quality and representativeness of blank samples; if blanks do not reflect true process contamination, intensity ratio may be misleading.98- Default threshold (ratio = 3) is data-dependent and may not suit all metabolite classes or instrumental platforms; users should validate via domain knowledge or pilot experiments.99- Method assumes intensity scales linearly with contamination; non-linear instrumental responses or matrix effects could bias the ratio.100- Exclusion of zero values before computing mean/median avoids division-by-zero but may artificially inflate ratios if blanks contain many missing values and unknowns contain sparse detections.101- README notes that support for GC and other data types is under development; currently documented for LC-MS only.102103## Evidence104105- [other] The blank_masking command removes features by comparing feature intensity in study samples (designated by sample_value) to intensity in blanks (designated by blank_value) using a configurable blank_intensity_ratio parameter; features whose intensity in unknown samples is not at least the specified ratio times more than blank samples are dropped.: "blank_masking command removes features by comparing feature intensity in study samples (designated by sample_value) to intensity in blanks (designated by blank_value) using a configurable"106- [other] For each feature, calculate the median or mean intensity across blank samples and across unknown samples, excluding zero values. Compute the intensity ratio (unknown/blank) for each feature and compare against the blank_intensity_ratio threshold (default 3). Retain only features where unknown sample intensity exceeds blank intensity by at least the threshold ratio.: "For each feature, calculate the median or mean intensity across blank samples and across unknown samples, excluding zero values. Compute the intensity ratio (unknown/blank) for each feature and"107- [other] Load the input feature table and sample metadata using the specified table moniker and identify blank and unknown samples based on the query field (e.g., 'sample_type').: "Load the input feature table and sample metadata using the specified table moniker and identify blank and unknown samples based on the query field (e.g., 'sample_type')"108- [other] Save the filtered feature table with a new moniker, preserving metadata and feature annotations.: "Save the filtered feature table with a new moniker, preserving metadata and feature annotations"109- [readme] The Python-Centric Pipeline for Metabolomics is designed to take raw LC-MS metabolomics data and ready them for downstream statistical analysis. The pipeline can perform quality control, data normalization and batch correction.: "perform quality control, data normalization and batch correction"