sample-metadata-stratification-and-filtering
License: restricted — no clear open-source license detected for the underlying tool; verify licensing before commercial use or redistribution.
Summary
Stratify and filter mass spectrometry samples by their metadata attributes (e.g., organism, sample type, geography) to enable targeted reanalysis and comparison of chemical annotations within ReDU. This skill reduces the scope of exploratory analysis by selecting cohesive subsets of public tandem MS data sharing common sample information properties.
When to use
When you have retrieved a large, heterogeneous collection of tandem MS files from ReDU or MassIVE and need to isolate a subset sharing specific sample characteristics (e.g., samples from a particular organism, body site, or geographic region) before computing PCA or comparing chemical annotations across a more homogeneous group.
When NOT to use
- You have only a small number of files (< 10) and manual curation is more practical than filtering logic.
- Your research question requires comparison across all known metabolomic diversity (unfiltered, global analysis) rather than within-group exploration.
- Sample metadata is sparse, incomplete, or not standardized according to the ReDU template — filtering will yield ambiguous or empty results.
Inputs
- ReDU database (populated with public MassIVE tandem MS files)
- Sample metadata attributes (organism, sample type, geography, tissue, body site, etc.)
- Validated ReDU sample information template entries for each file
Outputs
- Filtered subset of MassIVE file IDs meeting all active filter criteria
- List of chemical annotations (m/z, retention time, GNPS spectral match identifiers) for selected files
- Sample metadata table corresponding to filtered files (for downstream visualization or statistical analysis)
How to apply
Use the ReDU File Selector interface to apply filters via the Sample Information categories displayed as orange buttons in the center of the screen. Each filter criterion is applied cumulatively and appears as a red box in the Attribute Filters Panel (upper-right corner); the File Selector dynamically updates to show only files matching all active filters. Filters are based on validated sample metadata fields from the ReDU sample information template (e.g., organism, sample origin, sample type). Apply filters iteratively: start with broad stratification (e.g., organism), then narrow to secondary attributes (e.g., tissue or sample type) to balance statistical power with homogeneity. The resulting filtered file list becomes the input for downstream analyses such as PCA via chemical annotation matrix extraction or spectral library matching workflows.
Related tools
- ReDU (Web platform providing the File Selector UI, sample metadata index, and attribute filtering interface to query and subset public tandem MS files by sample information category) — https://github.com/mwang87/ReDU-MS2-GNPS
- MassIVE (Public data repository hosting raw and processed mass spectrometry datasets; ReDU indexes and filters MassIVE files via sample metadata)
- GNPS (Provides spectral library matching and chemical annotations (GNPS identifiers, confidence levels) that populate the chemical annotation matrix extracted for filtered file subsets) — https://github.com/CCMS-UCSD/GNPSDocumentation
- Emperor (Downstream visualization tool that accepts the filtered sample metadata and chemical annotation PCA results for interactive 3D/2D exploration and coloring by metadata stratification variable) — https://github.com/biocore/emperor
Evaluation signals
- The File Selector displays a reduced count of files matching all active filter criteria and shows each active filter as a red box in the Attribute Filters Panel (upper-right).
- Exported file list and metadata table contain only entries satisfying every applied filter condition (e.g., all samples are organism='Homo sapiens' AND sample_type='blood').
- The filtered chemical annotation matrix (rows=files, columns=features) contains no entries from files outside the filter criteria and has fewer rows than the unfiltered matrix.
- Emperor visualization of the filtered subset shows coherent clustering or separation by the stratification variable used for filtering, compared to noisy/dispersed patterns in unfiltered data.
- Downstream statistical tests (e.g., PERMANOVA on PCA scores) show stronger effect sizes or lower p-values when computed on the filtered, more homogeneous subset than on unfiltered data.
Limitations
- Metadata completeness and standardization varies across submitted datasets; some files may have missing or ambiguous sample information fields, reducing effective filter specificity.
- Over-stratification (too many sequential filters) may yield very small file counts (< 3–5), reducing statistical power for downstream analyses like PCA.
- Filtering is applied only to validated sample metadata; unmapped or non-standardized metadata fields cannot be used as filter criteria and may hide relevant sample attributes.
- The ReDU sample information template requires manual curation by data submitters; inconsistent terminology or abbreviations across datasets can lead to unexpected filter results despite seemingly identical filter selections.
Evidence
- [abstract] The orange buttons in the center of the screen correspond to Sample Information categories: "The orange buttons in the center of the screen correspond to Sample Information categories"
- [abstract] If filter/s are used, they will appear as red box/boxes in the Attribute Filters Panel (upper-right corner) of the page: "If filter/s are used, they will appear as red box/boxes in the Attribute Filters Panel (upper-right corner) of the page"
- [other] Query the ReDU database to retrieve chemical annotation matrices for selected MassIVE files, with rows as files and columns as annotated chemical features: "Query the ReDU database to retrieve chemical annotation matrices for selected MassIVE files, with rows as files and columns as annotated chemical features (m/z, retention time, GNPS annotations)"
- [methods] Fill in sample information using drop-downs when applicable: "Fill in sample information using drop-downs when applicable"
- [readme] ReDU is a community-minded approach to find and reuse public data containing tandem MS data at the repository scale: "ReDU is a community-minded approach to find and reuse public data containing tandem MS data at the repository scale. ReDU is a launchpad for co- or re-analysis of public data via the Global Natural"
1---2name: sample-metadata-stratification-and-filtering-23description: Use when when you have retrieved a large, heterogeneous collection of tandem MS files from ReDU or MassIVE and need to isolate a subset sharing specific sample characteristics (e.4license: CC-BY-4.05---67# sample-metadata-stratification-and-filtering89> **License: restricted** — no clear open-source license detected for the underlying tool; verify licensing before commercial use or redistribution. <!-- asb-license-banner -->10## Summary1112Stratify and filter mass spectrometry samples by their metadata attributes (e.g., organism, sample type, geography) to enable targeted reanalysis and comparison of chemical annotations within ReDU. This skill reduces the scope of exploratory analysis by selecting cohesive subsets of public tandem MS data sharing common sample information properties.1314## When to use1516When you have retrieved a large, heterogeneous collection of tandem MS files from ReDU or MassIVE and need to isolate a subset sharing specific sample characteristics (e.g., samples from a particular organism, body site, or geographic region) before computing PCA or comparing chemical annotations across a more homogeneous group.1718## When NOT to use1920- You have only a small number of files (< 10) and manual curation is more practical than filtering logic.21- Your research question requires comparison across all known metabolomic diversity (unfiltered, global analysis) rather than within-group exploration.22- Sample metadata is sparse, incomplete, or not standardized according to the ReDU template — filtering will yield ambiguous or empty results.2324## Inputs2526- ReDU database (populated with public MassIVE tandem MS files)27- Sample metadata attributes (organism, sample type, geography, tissue, body site, etc.)28- Validated ReDU sample information template entries for each file2930## Outputs3132- Filtered subset of MassIVE file IDs meeting all active filter criteria33- List of chemical annotations (m/z, retention time, GNPS spectral match identifiers) for selected files34- Sample metadata table corresponding to filtered files (for downstream visualization or statistical analysis)3536## How to apply3738Use the ReDU File Selector interface to apply filters via the Sample Information categories displayed as orange buttons in the center of the screen. Each filter criterion is applied cumulatively and appears as a red box in the Attribute Filters Panel (upper-right corner); the File Selector dynamically updates to show only files matching all active filters. Filters are based on validated sample metadata fields from the ReDU sample information template (e.g., organism, sample origin, sample type). Apply filters iteratively: start with broad stratification (e.g., organism), then narrow to secondary attributes (e.g., tissue or sample type) to balance statistical power with homogeneity. The resulting filtered file list becomes the input for downstream analyses such as PCA via chemical annotation matrix extraction or spectral library matching workflows.3940## Related tools4142- **ReDU** (Web platform providing the File Selector UI, sample metadata index, and attribute filtering interface to query and subset public tandem MS files by sample information category) — https://github.com/mwang87/ReDU-MS2-GNPS43- **MassIVE** (Public data repository hosting raw and processed mass spectrometry datasets; ReDU indexes and filters MassIVE files via sample metadata)44- **GNPS** (Provides spectral library matching and chemical annotations (GNPS identifiers, confidence levels) that populate the chemical annotation matrix extracted for filtered file subsets) — https://github.com/CCMS-UCSD/GNPSDocumentation45- **Emperor** (Downstream visualization tool that accepts the filtered sample metadata and chemical annotation PCA results for interactive 3D/2D exploration and coloring by metadata stratification variable) — https://github.com/biocore/emperor4647## Evaluation signals4849- The File Selector displays a reduced count of files matching all active filter criteria and shows each active filter as a red box in the Attribute Filters Panel (upper-right).50- Exported file list and metadata table contain only entries satisfying every applied filter condition (e.g., all samples are organism='Homo sapiens' AND sample_type='blood').51- The filtered chemical annotation matrix (rows=files, columns=features) contains no entries from files outside the filter criteria and has fewer rows than the unfiltered matrix.52- Emperor visualization of the filtered subset shows coherent clustering or separation by the stratification variable used for filtering, compared to noisy/dispersed patterns in unfiltered data.53- Downstream statistical tests (e.g., PERMANOVA on PCA scores) show stronger effect sizes or lower p-values when computed on the filtered, more homogeneous subset than on unfiltered data.5455## Limitations5657- Metadata completeness and standardization varies across submitted datasets; some files may have missing or ambiguous sample information fields, reducing effective filter specificity.58- Over-stratification (too many sequential filters) may yield very small file counts (< 3–5), reducing statistical power for downstream analyses like PCA.59- Filtering is applied only to validated sample metadata; unmapped or non-standardized metadata fields cannot be used as filter criteria and may hide relevant sample attributes.60- The ReDU sample information template requires manual curation by data submitters; inconsistent terminology or abbreviations across datasets can lead to unexpected filter results despite seemingly identical filter selections.6162## Evidence6364- [abstract] The orange buttons in the center of the screen correspond to Sample Information categories: "The orange buttons in the center of the screen correspond to Sample Information categories"65- [abstract] If filter/s are used, they will appear as red box/boxes in the Attribute Filters Panel (upper-right corner) of the page: "If filter/s are used, they will appear as red box/boxes in the Attribute Filters Panel (upper-right corner) of the page"66- [other] Query the ReDU database to retrieve chemical annotation matrices for selected MassIVE files, with rows as files and columns as annotated chemical features: "Query the ReDU database to retrieve chemical annotation matrices for selected MassIVE files, with rows as files and columns as annotated chemical features (m/z, retention time, GNPS annotations)"67- [methods] Fill in sample information using drop-downs when applicable: "Fill in sample information using drop-downs when applicable"68- [readme] ReDU is a community-minded approach to find and reuse public data containing tandem MS data at the repository scale: "ReDU is a community-minded approach to find and reuse public data containing tandem MS data at the repository scale. ReDU is a launchpad for co- or re-analysis of public data via the Global Natural"
HolobiomicsLab 