correlation-coefficient-computation-across-samples
Summary
Compute Pearson correlation coefficients between quantitative feature profiles across all samples to measure co-variation patterns. This metric is multiplied with retention time similarity to produce the total similarity score used for hierarchical clustering of metabolomic features.
When to use
After XCMS feature detection and retention time correction, when you need to group features derived from the same compound. Features from the same compound show correlated quantitative trends across samples; computing correlations between all feature pairs enables unsupervised discrimination of isotopic peaks, adducts, and fragments (which co-vary) from truly independent compounds (which do not).
When NOT to use
- Input is already a clustered or annotated feature table with compounds pre-defined; clustering should not be re-applied.
- Data contains only a single sample or very few replicates (<3); correlation estimates become unreliable with insufficient sample variation.
- Features show bimodal or multimodal intensity distributions across samples; Pearson correlation assumes linearity and may misrepresent association.
Inputs
- XCMS feature table (abundance matrix: samples × features, with retention time and m/z metadata)
- Feature quantitative profiles (normalized intensities across samples)
- Retention time similarity scores (0–1 normalized, pre-computed for each feature pair)
Outputs
- Pearson correlation coefficient matrix (features × features)
- Normalized correlational similarity matrix (0–1 scale, symmetric)
- Total similarity matrix (element-wise product of retention time and correlational similarity)
How to apply
For each pair of features in the XCMS-processed feature table, compute the Pearson correlation coefficient of their quantitative abundances across all samples. Each feature is represented as a vector of normalized intensities (one value per sample). Normalize the resulting correlation matrix to a 0–1 scale (e.g., by mapping [−1, 1] → [0, 1] or using absolute correlation). This correlational similarity score is then multiplied element-wise with the retention time similarity score (also normalized to 0–1) to produce the final total similarity matrix. Values near 1 indicate features likely derive from a single compound, while values near 0 indicate independent features. The resulting symmetric similarity matrix is input to hierarchical clustering (e.g., via dynamicTreeCut) to define feature clusters.
Related tools
- XCMS (Detects and aligns features from mass spectrometry data; provides input feature table with retention times and quantitative abundances)
- RAMClustR (Main clustering function that orchestrates retention time similarity, correlational similarity computation, and hierarchical clustering) — https://github.com/cbroeckl/RAMClustR
- dynamicTreeCut (Receives the total similarity matrix and cuts the resulting dendrogram into feature clusters)
- R (Execution environment; Pearson correlation computation via base R or tidyverse functions)
Examples
# Within R after loading XCMS object and computing retention time similarity
cor_matrix <- cor(t(featureTable)) # Pearson correlation across samples
cor_sim <- (cor_matrix + 1) / 2 # Normalize [-1, 1] to [0, 1]
total_sim <- rt_similarity * cor_sim # Element-wise product
RC <- ramclustR(xcmsObj = xset, ExpDes = experiment) # Full workflow including correlation computation
Evaluation signals
- Correlation matrix is symmetric (corr[i,j] == corr[j,i]) and diagonal elements equal 1.0 (self-correlation).
- Normalized similarity scores fall within [0, 1]; no NaN or infinite values present.
- Features from the same known compound (e.g., isotopic variants or adducts in test data) produce correlational similarity > 0.8; unrelated features produce values < 0.3.
- Total similarity matrix (product of retention time and correlational similarities) yields feature clusters consistent with reference standards or manual inspection of mass spectra.
- Histogram of similarity scores shows bimodal distribution (peaks near 0 and near 1), indicating clear separation between same-compound and different-compound feature pairs.
Limitations
- Pearson correlation assumes linear relationships; non-linear co-variation (e.g., saturation, sigmoidal responses) may be misclassified.
- With few samples or high missing-value proportions, correlation estimates are unstable; fillPeaks preprocessing is essential.
- Sporadic features present in only one or two samples will have undefined or unreliable correlations with other features.
- Correlational similarity alone cannot distinguish features from isobaric compounds that happen to have similar sample profiles; retention time similarity is necessary to resolve this ambiguity.
- Batch effects and systematic drift (not corrected by retcor) can inflate spurious correlations between unrelated features across samples.
Evidence
- [other] Compute Pearson correlation coefficients between quantitative feature profiles across all samples to obtain correlational similarity scores.: "Compute Pearson correlation coefficients between quantitative feature profiles across all samples to obtain correlational similarity scores."
- [intro] two features derived from the same compound will have (approximately) the same [quantitative trend across samples]: "two features derived from the same compound will have (approximately) the same"
- [intro] Since both conditions must be met, the product of the two similarity scores provides the best approximatio of the total similarity score: "Since both conditions must be met, the product of the two similarity scores provides the best approximatio of the total similarity score"
- [intro] submitting this score matrix for heirarchical clustering, and then cutting the resulting dendrogram into neat chunks using the dynamicTreeCut package: "submitting this score matrix for heirarchical clustering, and then cutting the resulting dendrogram into neat chunks using the dynamicTreeCut package"
- [other] RAMClustR calculates a pairwise total similarity score as the product of retention time similarity and correlational similarity scores.: "RAMClustR calculates a pairwise total similarity score as the product of retention time similarity and correlational similarity scores."
1---2name: correlation-coefficient-computation-across-samples3description: Use when after XCMS feature detection and retention time correction, when you need to group features derived from the same compound. Features from the same compound show correlated quantitative trends across samples;4license: CC-BY-4.05---67# correlation-coefficient-computation-across-samples89## Summary1011Compute Pearson correlation coefficients between quantitative feature profiles across all samples to measure co-variation patterns. This metric is multiplied with retention time similarity to produce the total similarity score used for hierarchical clustering of metabolomic features.1213## When to use1415After XCMS feature detection and retention time correction, when you need to group features derived from the same compound. Features from the same compound show correlated quantitative trends across samples; computing correlations between all feature pairs enables unsupervised discrimination of isotopic peaks, adducts, and fragments (which co-vary) from truly independent compounds (which do not).1617## When NOT to use1819- Input is already a clustered or annotated feature table with compounds pre-defined; clustering should not be re-applied.20- Data contains only a single sample or very few replicates (<3); correlation estimates become unreliable with insufficient sample variation.21- Features show bimodal or multimodal intensity distributions across samples; Pearson correlation assumes linearity and may misrepresent association.2223## Inputs2425- XCMS feature table (abundance matrix: samples × features, with retention time and m/z metadata)26- Feature quantitative profiles (normalized intensities across samples)27- Retention time similarity scores (0–1 normalized, pre-computed for each feature pair)2829## Outputs3031- Pearson correlation coefficient matrix (features × features)32- Normalized correlational similarity matrix (0–1 scale, symmetric)33- Total similarity matrix (element-wise product of retention time and correlational similarity)3435## How to apply3637For each pair of features in the XCMS-processed feature table, compute the Pearson correlation coefficient of their quantitative abundances across all samples. Each feature is represented as a vector of normalized intensities (one value per sample). Normalize the resulting correlation matrix to a 0–1 scale (e.g., by mapping [−1, 1] → [0, 1] or using absolute correlation). This correlational similarity score is then multiplied element-wise with the retention time similarity score (also normalized to 0–1) to produce the final total similarity matrix. Values near 1 indicate features likely derive from a single compound, while values near 0 indicate independent features. The resulting symmetric similarity matrix is input to hierarchical clustering (e.g., via dynamicTreeCut) to define feature clusters.3839## Related tools4041- **XCMS** (Detects and aligns features from mass spectrometry data; provides input feature table with retention times and quantitative abundances)42- **RAMClustR** (Main clustering function that orchestrates retention time similarity, correlational similarity computation, and hierarchical clustering) — https://github.com/cbroeckl/RAMClustR43- **dynamicTreeCut** (Receives the total similarity matrix and cuts the resulting dendrogram into feature clusters)44- **R** (Execution environment; Pearson correlation computation via base R or tidyverse functions)4546## Examples4748```49# Within R after loading XCMS object and computing retention time similarity50cor_matrix <- cor(t(featureTable)) # Pearson correlation across samples51cor_sim <- (cor_matrix + 1) / 2 # Normalize [-1, 1] to [0, 1]52total_sim <- rt_similarity * cor_sim # Element-wise product53RC <- ramclustR(xcmsObj = xset, ExpDes = experiment) # Full workflow including correlation computation54```5556## Evaluation signals5758- Correlation matrix is symmetric (corr[i,j] == corr[j,i]) and diagonal elements equal 1.0 (self-correlation).59- Normalized similarity scores fall within [0, 1]; no NaN or infinite values present.60- Features from the same known compound (e.g., isotopic variants or adducts in test data) produce correlational similarity > 0.8; unrelated features produce values < 0.3.61- Total similarity matrix (product of retention time and correlational similarities) yields feature clusters consistent with reference standards or manual inspection of mass spectra.62- Histogram of similarity scores shows bimodal distribution (peaks near 0 and near 1), indicating clear separation between same-compound and different-compound feature pairs.6364## Limitations6566- Pearson correlation assumes linear relationships; non-linear co-variation (e.g., saturation, sigmoidal responses) may be misclassified.67- With few samples or high missing-value proportions, correlation estimates are unstable; fillPeaks preprocessing is essential.68- Sporadic features present in only one or two samples will have undefined or unreliable correlations with other features.69- Correlational similarity alone cannot distinguish features from isobaric compounds that happen to have similar sample profiles; retention time similarity is necessary to resolve this ambiguity.70- Batch effects and systematic drift (not corrected by retcor) can inflate spurious correlations between unrelated features across samples.7172## Evidence7374- [other] Compute Pearson correlation coefficients between quantitative feature profiles across all samples to obtain correlational similarity scores.: "Compute Pearson correlation coefficients between quantitative feature profiles across all samples to obtain correlational similarity scores."75- [intro] two features derived from the same compound will have (approximately) the same [quantitative trend across samples]: "two features derived from the same compound will have (approximately) the same"76- [intro] Since both conditions must be met, the product of the two similarity scores provides the best approximatio of the total similarity score: "Since both conditions must be met, the product of the two similarity scores provides the best approximatio of the total similarity score"77- [intro] submitting this score matrix for heirarchical clustering, and then cutting the resulting dendrogram into neat chunks using the dynamicTreeCut package: "submitting this score matrix for heirarchical clustering, and then cutting the resulting dendrogram into neat chunks using the dynamicTreeCut package"78- [other] RAMClustR calculates a pairwise total similarity score as the product of retention time similarity and correlational similarity scores.: "RAMClustR calculates a pairwise total similarity score as the product of retention time similarity and correlational similarity scores."
HolobiomicsLab