# Formula Deduplication And Counting

> Use when when you have loaded a chemical database (e.g., HMDB pickle file) and need to understand how many distinct molecular formulas remain after filtering for a specific m/z range (e.g., 100–1000) and MS acquisition level (typically MS level 1).

- Skill: `holobiomicslab/formula-deduplication-and-counting` (Agent Skill)
- Install (CLI): `npx skillmds@latest add holobiomicslab/formula-deduplication-and-counting`
- Raw SKILL.md: https://api.skillmd.com/api/skills/holobiomicslab/formula-deduplication-and-counting/raw
- Safety review: PASS (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- License: CC-BY-4.0
- Author: HolobiomicsLab (https://skillmd.com/u/holobiomicslab)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/holobiomicslab/formula-deduplication-and-counting

---


# formula-deduplication-and-counting

## Summary

Deduplicate and count unique molecular formulas from a filtered chemical database after applying m/z range and MS level constraints. This skill quantifies the diversity of ionizable species available in a virtual metabolomics acquisition simulation.

## When to use

When you have loaded a chemical database (e.g., HMDB pickle file) and need to understand how many distinct molecular formulas remain after filtering for a specific m/z range (e.g., 100–1000) and MS acquisition level (typically MS level 1). This is essential before running virtual mass spectrometry experiments to ensure the chemical space is correctly scoped.

## When NOT to use

- If the input database is already in a feature table or has been pre-deduplicated and you only need to query the count without re-filtering.
- If you need to preserve or analyze duplicate formula entries (e.g., to study isotopologues or different adducts of the same formula).
- If your workflow requires formula-level metadata (e.g., retention time, intensity, or adduct annotation) — this skill only counts unique formulas.

## Inputs

- HMDB compounds database (pickled Python object, .p file)
- m/z range specification (min_mz, max_mz)
- MS level filter parameter (e.g., ms_levels=1)

## Outputs

- Deduplicated molecular formula list (set or pandas Series)
- Count of unique molecular formulas (integer)
- Summary report with formula count and filter parameters

## How to apply

Load the HMDB compounds database using Python pickle deserialization. Apply a minimum and maximum m/z filter (e.g., min_mz=100, max_mz=1000) to retain only formulas within the instrument's observable range. Further filter to retain only compounds with ms_levels=1 to match the target acquisition mode. Deduplicate the filtered formula set by converting to a set or using pandas drop_duplicates() to remove duplicate entries. Count the total number of unique formulas and record in a summary report or return as a scalar for downstream validation or comparisons.

## Related tools

- **ViMMS** (Framework within which the formula sampling and filtering occurs; DatabaseFormulaSampler applies the m/z and MS level filters before deduplication) — https://github.com/glasgowcompbio/vimms
- **Python pickle** (Deserializes the HMDB compounds database (.p file) into memory for filtering)
- **pandas** (Optional utility for efficient deduplication and counting of formulas)

## Examples

```
import pickle
with open('hmdb_compounds.p', 'rb') as f:
    hmdb = pickle.load(f)
filtered = [f for f in hmdb if 100 <= f.mass <= 1000 and f.ms_levels == 1]
unique_formulas = set(f.formula for f in filtered)
print(len(unique_formulas))
```

## Evaluation signals

- The count of unique formulas equals the cardinality of the deduplicated set (no duplicates remain).
- The reported count (73,822 for HMDB with m/z 100–1000, MS level 1) is reproducible across multiple runs with identical filter parameters.
- All returned formulas have m/z values within the specified range (100–1000) when calculated from their neutral masses.
- Formulas excluded by the m/z or MS level filter are not present in the final deduplicated set.
- The count is ≥ 1 and ≤ the total number of formulas in the original database (sanity bounds check).

## Limitations

- The skill depends on the completeness and accuracy of the input HMDB database file; corrupt or incomplete .p files will yield incorrect counts.
- M/z calculations assume a specific ionization mode (implied but not explicitly stated in the workflow); switching ionization modes (positive vs. negative) may change the effective m/z range and thus the count.
- Deduplication is formula-level only; structurally different compounds with identical molecular formulas are not distinguished and will not be double-counted.
- The skill does not validate or flag ambiguous or unusual formulas; implausible elemental compositions will still be counted.

## Evidence

- [other] When the DatabaseFormulaSampler filters the HMDB database for formulas with m/z between 100 and 1000, it yields 73822 unique molecular formulas.: "When the DatabaseFormulaSampler filters the HMDB database for formulas with m/z between 100 and 1000, it yields 73822 unique molecular formulas."
- [other] Apply m/z range filter (min_mz=100, max_mz=1000) to retain only formulas within the specified range. Filter to retain only MS level 1 compounds by selecting those with ms_levels=1. De-duplicate the filtered formula set to obtain unique molecular formula entries.: "Apply m/z range filter (min_mz=100, max_mz=1000) to retain only formulas within the specified range. 3. Filter to retain only MS level 1 compounds by selecting those with ms_levels=1. 4. De-duplicate"
- [other] Load the HMDB compounds database (hmdb_compounds.p) using Python pickle.: "Load the HMDB compounds database (hmdb_compounds.p) using Python pickle."
- [other] Chemicals can be sampled from databases such as HMDB or from specific distributions: "Chemicals can be sampled from databases such as HMDB or from specific distributions"