# Feature Missingness Threshold Filtering

> Use when after loading and formatting raw peak-picked LC-MS metabolomics data frames (via metabData constructor) when you need to eliminate features with poor sample coverage before feature alignment.

- Skill: `holobiomicslab/feature-missingness-threshold-filtering` (Agent Skill)
- Install (CLI): `npx skillmds@latest add holobiomicslab/feature-missingness-threshold-filtering`
- Raw SKILL.md: https://api.skillmd.com/api/skills/holobiomicslab/feature-missingness-threshold-filtering/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- License: CC-BY-4.0
- Author: HolobiomicsLab (https://skillmd.com/u/holobiomicslab)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/holobiomicslab/feature-missingness-threshold-filtering

---


# feature-missingness-threshold-filtering

## Summary

Remove low-abundance or sparsely-detected metabolomic features from LC-MS datasets by applying a sample-wise missingness threshold. This filtering step eliminates features detected in fewer than a specified percentage of samples, reducing noise and improving feature quality before downstream alignment and scoring.

## When to use

Apply this skill after loading and formatting raw peak-picked LC-MS metabolomics data frames (via metabData constructor) when you need to eliminate features with poor sample coverage before feature alignment. Triggers: (1) raw peak table contains features with sporadic detection across replicates or sample groups; (2) you aim to reduce false positives and focus on robust, reproducibly-detected metabolites; (3) downstream alignment or scoring steps are sensitive to feature sparsity.

## When NOT to use

- Input is already a curated, published feature table with hand-verified annotations (missingness filtering may discard biologically-relevant rare features).
- Experimental design intentionally includes sparse, low-abundance metabolites as biological endpoints (e.g., disease biomarkers detected in only 10% of samples; strict missingness filtering would eliminate them).
- Data has already been filtered for missingness upstream; applying redundant thresholds risks over-filtering and loss of reproducible signal.

## Inputs

- raw peak-picked LC-MS data frame with mz, rt, id, adduct, and sample columns
- misspc parameter: numeric threshold (0–100) indicating minimum sample detection percentage

## Outputs

- metabData object: formatted and filtered feature table with features below misspc threshold removed
- retained feature count: number of features passing missingness filter

## How to apply

Within the metabData formatting workflow, invoke the missingness filter by specifying the `misspc` argument (a percentage threshold, e.g., 50%). The filter scans all sample columns for each feature and counts the proportion of non-missing values (detections). Features whose detection frequency falls below the `misspc` threshold are removed from the formatted metabData object. The rationale is that features present in only a small fraction of samples are likely artifacts, instrument noise, or biological outliers; retaining only features meeting the threshold ensures robust m/z and retention time mappings for subsequent pairwise alignment. Typical thresholds range from 50–80% depending on experimental design (e.g., loose for exploratory work, strict for targeted follow-up).

## Related tools

- **metabCombiner** (R package providing metabData constructor and missingness filter implementation for LC-MS metabolomics data formatting and filtering) — https://github.com/hhabra/metabCombiner
- **R** (execution environment for metabCombiner and data frame manipulation)

## Examples

```
metabData(plasma20, mz='mz', rt='rt', id='id', adduct='adduct', samples='CHEAR', misspc=50)
```

## Evaluation signals

- Verify that all retained features have detection frequency ≥ misspc threshold across sample columns; audit marginal cases near threshold boundary.
- Compare feature count before and after filtering; report percentage of features eliminated and confirm it aligns with expected sparsity of input dataset.
- Inspect downstream m/z grouping and pairwise alignment step: if missingness filtering succeeded, feature-pair detection should show stable, non-spurious alignments without artefactual m/z clustering.
- Confirm that no sample columns are entirely empty (all-missing) in the output metabData; presence of such a column indicates filter logic error.
- Cross-check that the misspc parameter used matches the analysis plan; missingness filter should be applied only once per dataset to avoid cascade effects.

## Limitations

- Missingness threshold is global across all sample columns; the filter does not account for sample-group-specific sparsity patterns (e.g., biomarkers present only in disease subgroup may be filtered if they are rare in the entire cohort).
- Filter operates on detection (non-missing) status only; it does not consider intensity or signal-to-noise ratio, so a feature detected in many samples at very low abundance may pass the threshold but be uninformative.
- Threshold choice (misspc value) is user-defined and not data-driven; no principled guidance is provided in the article for selecting optimal thresholds for different experimental designs.
- Filtering is irreversible within the metabData object; users must retain the original data frame if downstream reanalysis with different thresholds is anticipated.

## Evidence

- [intro] The missingness filter eliminates features below some threshold percentage indicated by the *misspc* argument.: "The missingness filter eliminates features below some threshold percentage indicated by the *misspc* argument."
- [methods] For samples and extra fields, metabData searches for all columns containing any of the keywords in the respective arguments, with regular expressions accepted for any indicated field.: "for samples and extra fields, [metabData] searches for all columns containing any of the keywords in the respective arguments, with regular expressions accepted for any indicated field."
- [intro] metabCombiner determines possible feature pair alignments and validates them through pairwise similarity scoring.: "`metabCombiner` determines a list possible feature pair alignments (FPAs) and determines their validity through a pairwise similarity score."