# Technical Heterogeneity Removal

> Use when your metabolomics matrix (log2-scaled, samples × features in .csv format) exhibits spatial or distributional separation by batch/analytical run, visible in PCA or RLA plots.

- Skill: `holobiomicslab/technical-heterogeneity-removal` (Agent Skill)
- Install (CLI): `npx skillmds@latest add holobiomicslab/technical-heterogeneity-removal`
- Raw SKILL.md: https://api.skillmd.com/api/skills/holobiomicslab/technical-heterogeneity-removal/raw
- Safety review: PASS (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Data & Analytics
- License: CC-BY-4.0
- Author: HolobiomicsLab (https://skillmd.com/u/holobiomicslab)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/holobiomicslab/technical-heterogeneity-removal

---


# Technical Heterogeneity Removal from Metabolomics Datasets

> **License: restricted** — no clear open-source license detected for the underlying tool; verify licensing before commercial use or redistribution. <!-- asb-license-banner -->
## Summary

This skill removes batch effects and drift across analytical runs in metabolomics datasets using statistical correction models (two-stage procedure, parametric/non-parametric ComBat, or bagging-based regression). It is essential when metabolomics data span multiple batches or time periods and technical variation threatens the validity of biological inference.

## When to use

Apply this skill when your metabolomics matrix (log2-scaled, samples × features in .csv format) exhibits spatial or distributional separation by batch/analytical run, visible in PCA or RLA plots. Use it after missing-value imputation and before differential abundance or association testing, especially if batches contain identical quality-control replicates that should cluster together regardless of analytical run.

## When NOT to use

- Data are not log2-scaled or do not account for high-abundance feature dominance; preprocessing must precede correction.
- Single batch or no evidence of batch effect (PCA/RLA shows no batch clustering); correction is unnecessary and risks overfitting.
- Feature count > 2000 and computational speed is critical; Visodbnorm and dbnormSCORE functions will be slow and are not recommended by the authors.

## Inputs

- Metabolomics data matrix (samples × features, .csv format)
- Log2-scaled intensity or abundance values
- Batch/analytical-run identifiers (first column)
- Missing values imputed (zero or NA replaced with feature or global minimum)

## Outputs

- Batch-corrected metabolomics matrix (.csv file)
- Adjusted R² scores for each feature and model comparison table (.csv)
- Diagnostic plots (PCA, scree, RLA, correlation, probability density function profiles) in PDF
- Hierarchical clustering dendrograms of original vs. corrected data (optional)

## How to apply

Load log2-scaled, batch-annotated metabolomics data (first column = batch identifier, remaining columns = features). Choose a batch correction model: the two-stage procedure (ber function) for microarray-derived drift patterns, parametric ComBat for normally distributed features, non-parametric ComBat for skewed distributions, or ber-bagging for robust partial bootstrap correction (n=150 samples). Run the corresponding dbnorm function (dbnormBer, dbnormPcom, dbnormNPcom, or dbnormBagging). Evaluate model fit using dbnormSCORE, which calculates adjusted R² for each feature's dependence on batch before and after correction; select the model maximizing consistency across features. Generate diagnostic plots (PCA, RLA, correlation, distribution profiles) to confirm batch signal removal without introducing artificial clustering.

## Related tools

- **dbnorm** (Primary R package providing 11 functions for missing-value imputation, batch correction (ber, ComBat variants, bagging), and diagnostic visualization for metabolomics drift normalization) — https://github.com/NBDZ/dbnorm
- **sva** (Source of parametric and non-parametric ComBat empirical Bayes methods, wrapped and extended in dbnorm for metabolomics use)
- **ber** (Two-stage batch-effect regression model from microarray literature, ported to metabolomics via dbnorm)

## Examples

```
data <- read.csv('metabolomics_raw_log2.csv', sep=',', header=TRUE, row.names=1)
library(dbnorm)
dbnormSCORE(data)  # Evaluate model fit
dbnormBer(data)    # Apply two-stage batch correction and generate diagnostics
```

## Evaluation signals

- Adjusted R² for each feature decreases after correction, indicating reduced batch dependence; check dbnormSCORE output table for consistency across features.
- PCA and RLA plots show elimination of batch-driven clustering or drift; samples from the same biological group cluster together regardless of batch assignment.
- Hierarchical clustering of quality-control replicates analyzed in different batches shows reduced dissimilarity (Pearson distance, average linkage) post-correction compared to raw data.
- Probability density function plots (ProfPlot* functions) show alignment of feature distributions across batches, with no artificial multimodality or distortion.
- No systematic loss of biological signal: effect sizes for known biological associations (if available) remain unchanged or improve; feature variance explained by non-batch covariates is preserved.

## Limitations

- Requires batch-level annotation in first column of input matrix; missing or mislabeled batch assignments will cause incorrect correction.
- Assumes linear batch effects; non-linear drift or instrument malfunction in a single run may not be fully captured.
- Parametric ComBat assumes approximate normality; highly skewed features may require log transformation or non-parametric variant.
- Bagging approach (ber-bagging) uses partial bootstrap (n=150); convergence and stability for very small sample counts (n < 30 per batch) not reported.
- No automatic model selection; users must interpret diagnostic plots and adjusted R² scores to choose between ber, bagging, and ComBat variants—suboptimal choice risks over- or under-correction.
- Performance degrades for > 2000 features; computational speed not reported for large feature sets.

## Evidence

- [readme] dbnorm includes 11 distinct functions for pre-processing of data and estimation of missing values, conventional functions for batch effect correction based on statistical models: "dbnorm includes 11 distinct functions for pre-processing of data and estimation of missing values, conventional functions for batch effect correction based on statistical models"
- [readme] Data to be uploaded must be normalized and scaled on the log2 to account for the high abundance features (variables) by which technical heterogeneity might be overlooked.: "Data to be uploaded must be normalized and scaled on the log2 to account for the high abundance features (variables) by which technical heterogeneity might be overlooked."
- [intro] It allows users to efficiently correct drift across batch and to adjust large metabolomics datasets for technical variation which helps improving the estimation of the biological mechanisms underlying disease condition: "It allows users to efficiently correct drift across batch and to adjust large metabolomics datasets for technical variation which helps improving the estimation"
- [readme] Adjusted R-squared is calculated for each variable estimated in a regression model for its dependency to the batch level in the raw data and treated data via either of those models.: "Adjusted R-squared is calculated for each variable estimated in a regression model for its dependency to the batch level in the raw data and treated data"
- [readme] This function is suggested for less than 2000 features (variables) for better computational speed.: "This function is suggested for less than 2000 features (variables) for better computational speed."