hierarchical-clustering-parameter-optimization
Summary
Optimize hierarchical clustering parameters (euclidean distance metric, drift time and retention time spread thresholds) to assign detected GC-IMS peaks to reproducible clusters across samples. This skill determines the correct distance metric and spread tolerances needed to group peaks with similar chromatographic and ion mobility properties.
When to use
After peak detection in GC-IMS preprocessing, when you need to group peaks across multiple samples and must decide whether euclidean distance is appropriate for your drift time and retention time coordinate space, and when you need to validate that your chosen dt_cluster_spread_ms and rt_cluster_spread_s thresholds produce reproducible, biologically meaningful cluster assignments that can be compared across replicate samples.
When NOT to use
- Input peaks are already pre-clustered or annotated with compound identity — re-clustering may destroy valid prior assignments.
- Drift time or retention time data have not been aligned across samples — misalignments will confound cluster assignments.
- Sample mixtures contain compounds with overlapping or identical retention and drift times; euclidean distance alone may not resolve them without additional chemical information.
Inputs
- peak detection results (peaks with drift time and retention time coordinates)
- peak list with drift_time and retention_time attributes
Outputs
- peak_clustering result object
- peak_list_clustered with cluster IDs assigned to each peak
- cluster_stats with median cluster positions and cluster membership counts
How to apply
Load peak detection results containing drift time and retention time coordinates for each detected peak. Apply the clusterPeaks function from GCIMS with distance_method='euclidean', setting dt_cluster_spread_ms=0.1 (drift time tolerance in milliseconds) and rt_cluster_spread_s=20 (retention time tolerance in seconds) as initial parameters, and specify hclust as the linkage method. The euclidean distance metric combines drift time and retention time differences into a single dissimilarity measure; the spread thresholds define the maximum within-cluster variance allowed in each dimension. Extract the resulting cluster IDs from peak_list_clustered and inspect cluster_stats (median cluster positions) to verify that clusters are compact and separated. Adjust spread thresholds downward if clusters are too loose (merging distinct compounds) or upward if valid peaks are over-fragmented across clusters.
Related tools
- GCIMS (Provides the clusterPeaks function and peak_clustering data structures for hierarchical clustering with euclidean distance and hclust linkage.) — https://github.com/sipss/GCIMS
- R (Execution environment for GCIMS and hierarchical clustering operations.)
Examples
peak_clustering <- clusterPeaks(peaks, distance_method='euclidean', dt_cluster_spread_ms=0.1, rt_cluster_spread_s=20, linkage_method='hclust')
Evaluation signals
- Cluster IDs are assigned consistently to all peaks in peak_list_clustered without missing or invalid values.
- Median cluster positions in cluster_stats fall within the specified spread thresholds (euclidean distance ≤ threshold defined by dt_cluster_spread_ms and rt_cluster_spread_s).
- Cluster assignments are reproducible when re-run on the same peak set and parameters.
- Visual inspection of 2D scatter plots (drift time vs. retention time, colored by cluster ID) shows compact, well-separated clusters without obvious over-merging or fragmentation.
- Cluster sizes are stable across replicate samples; biologically expected compounds appear in the same clusters in replicate injections.
Limitations
- Euclidean distance assumes drift time and retention time are on commensurate scales; if they differ by orders of magnitude, the metric may be dominated by one dimension and require data normalization.
- Fixed spread thresholds (dt_cluster_spread_ms=0.1, rt_cluster_spread_s=20) may not be optimal for all sample types or chromatographic methods; validation on your specific instrument and column is essential.
- Hierarchical clustering is sensitive to linkage method; the article specifies hclust but does not justify why it is superior to other linkage methods (e.g., complete, average, or Ward) for this application.
- No guidance is provided for tuning spread thresholds a priori; the article shows one successful case (three ketones dataset) but does not describe how to choose thresholds for novel compound mixtures.
Evidence
- [other] clusterPeaks with euclidean distance, dt_cluster_spread_ms=0.1, rt_cluster_spread_s=20, and hclust method produces peak_clustering output containing peak_list_clustered with cluster IDs and cluster_stats with median cluster positions.: "clusterPeaks with euclidean distance, dt_cluster_spread_ms=0.1, rt_cluster_spread_s=20, and hclust method produces peak_clustering output containing peak_list_clustered with cluster IDs and"
- [other] What are the cluster assignments for detected peaks when using hierarchical clustering with euclidean distance and specified drift time and retention time spread thresholds?: "cluster assignments for detected peaks when using hierarchical clustering with euclidean distance and specified drift time and retention time spread thresholds"
- [other] Load the peak detection results (peaks with drift time and retention time coordinates). 2. Apply clusterPeaks function from GCIMS with distance_method='euclidean', dt_cluster_spread_ms=0.1, rt_cluster_spread_s=20, and hclust linkage method.: "Load the peak detection results (peaks with drift time and retention time coordinates). Apply clusterPeaks function from GCIMS with distance_method='euclidean', dt_cluster_spread_ms=0.1,"
- [readme] GCIMS is an R package implementing a preprocessing pipeline for Gas Chromatography – Ion Mobility Spectrometry samples.: "GCIMS is an R package implementing a preprocessing pipeline for Gas Chromatography – Ion Mobility Spectrometry samples."
- [intro] Pressure and temperature fluctuations as well as degradation of the chromatographic column are some of the causes of misalignments in the data, both in retention and drift time.: "Pressure and temperature fluctuations as well as degradation of the chromatographic column are some of the causes of misalignments in the data, both in retention and drift time."
1---2name: hierarchical-clustering-parameter-optimization-23description: Use when after peak detection in GC-IMS preprocessing, when you need to group peaks across multiple samples and must decide whether euclidean distance is appropriate for your drift time and retention time coordinate space, and when you need to validate that your chosen dt_cluster_spread_ms and.4license: CC-BY-4.05---67# hierarchical-clustering-parameter-optimization89## Summary1011Optimize hierarchical clustering parameters (euclidean distance metric, drift time and retention time spread thresholds) to assign detected GC-IMS peaks to reproducible clusters across samples. This skill determines the correct distance metric and spread tolerances needed to group peaks with similar chromatographic and ion mobility properties.1213## When to use1415After peak detection in GC-IMS preprocessing, when you need to group peaks across multiple samples and must decide whether euclidean distance is appropriate for your drift time and retention time coordinate space, and when you need to validate that your chosen dt_cluster_spread_ms and rt_cluster_spread_s thresholds produce reproducible, biologically meaningful cluster assignments that can be compared across replicate samples.1617## When NOT to use1819- Input peaks are already pre-clustered or annotated with compound identity — re-clustering may destroy valid prior assignments.20- Drift time or retention time data have not been aligned across samples — misalignments will confound cluster assignments.21- Sample mixtures contain compounds with overlapping or identical retention and drift times; euclidean distance alone may not resolve them without additional chemical information.2223## Inputs2425- peak detection results (peaks with drift time and retention time coordinates)26- peak list with drift_time and retention_time attributes2728## Outputs2930- peak_clustering result object31- peak_list_clustered with cluster IDs assigned to each peak32- cluster_stats with median cluster positions and cluster membership counts3334## How to apply3536Load peak detection results containing drift time and retention time coordinates for each detected peak. Apply the clusterPeaks function from GCIMS with distance_method='euclidean', setting dt_cluster_spread_ms=0.1 (drift time tolerance in milliseconds) and rt_cluster_spread_s=20 (retention time tolerance in seconds) as initial parameters, and specify hclust as the linkage method. The euclidean distance metric combines drift time and retention time differences into a single dissimilarity measure; the spread thresholds define the maximum within-cluster variance allowed in each dimension. Extract the resulting cluster IDs from peak_list_clustered and inspect cluster_stats (median cluster positions) to verify that clusters are compact and separated. Adjust spread thresholds downward if clusters are too loose (merging distinct compounds) or upward if valid peaks are over-fragmented across clusters.3738## Related tools3940- **GCIMS** (Provides the clusterPeaks function and peak_clustering data structures for hierarchical clustering with euclidean distance and hclust linkage.) — https://github.com/sipss/GCIMS41- **R** (Execution environment for GCIMS and hierarchical clustering operations.)4243## Examples4445```46peak_clustering <- clusterPeaks(peaks, distance_method='euclidean', dt_cluster_spread_ms=0.1, rt_cluster_spread_s=20, linkage_method='hclust')47```4849## Evaluation signals5051- Cluster IDs are assigned consistently to all peaks in peak_list_clustered without missing or invalid values.52- Median cluster positions in cluster_stats fall within the specified spread thresholds (euclidean distance ≤ threshold defined by dt_cluster_spread_ms and rt_cluster_spread_s).53- Cluster assignments are reproducible when re-run on the same peak set and parameters.54- Visual inspection of 2D scatter plots (drift time vs. retention time, colored by cluster ID) shows compact, well-separated clusters without obvious over-merging or fragmentation.55- Cluster sizes are stable across replicate samples; biologically expected compounds appear in the same clusters in replicate injections.5657## Limitations5859- Euclidean distance assumes drift time and retention time are on commensurate scales; if they differ by orders of magnitude, the metric may be dominated by one dimension and require data normalization.60- Fixed spread thresholds (dt_cluster_spread_ms=0.1, rt_cluster_spread_s=20) may not be optimal for all sample types or chromatographic methods; validation on your specific instrument and column is essential.61- Hierarchical clustering is sensitive to linkage method; the article specifies hclust but does not justify why it is superior to other linkage methods (e.g., complete, average, or Ward) for this application.62- No guidance is provided for tuning spread thresholds a priori; the article shows one successful case (three ketones dataset) but does not describe how to choose thresholds for novel compound mixtures.6364## Evidence6566- [other] clusterPeaks with euclidean distance, dt_cluster_spread_ms=0.1, rt_cluster_spread_s=20, and hclust method produces peak_clustering output containing peak_list_clustered with cluster IDs and cluster_stats with median cluster positions.: "clusterPeaks with euclidean distance, dt_cluster_spread_ms=0.1, rt_cluster_spread_s=20, and hclust method produces peak_clustering output containing peak_list_clustered with cluster IDs and"67- [other] What are the cluster assignments for detected peaks when using hierarchical clustering with euclidean distance and specified drift time and retention time spread thresholds?: "cluster assignments for detected peaks when using hierarchical clustering with euclidean distance and specified drift time and retention time spread thresholds"68- [other] Load the peak detection results (peaks with drift time and retention time coordinates). 2. Apply clusterPeaks function from GCIMS with distance_method='euclidean', dt_cluster_spread_ms=0.1, rt_cluster_spread_s=20, and hclust linkage method.: "Load the peak detection results (peaks with drift time and retention time coordinates). Apply clusterPeaks function from GCIMS with distance_method='euclidean', dt_cluster_spread_ms=0.1,"69- [readme] GCIMS is an R package implementing a preprocessing pipeline for Gas Chromatography – Ion Mobility Spectrometry samples.: "GCIMS is an R package implementing a preprocessing pipeline for Gas Chromatography – Ion Mobility Spectrometry samples."70- [intro] Pressure and temperature fluctuations as well as degradation of the chromatographic column are some of the causes of misalignments in the data, both in retention and drift time.: "Pressure and temperature fluctuations as well as degradation of the chromatographic column are some of the causes of misalignments in the data, both in retention and drift time."
HolobiomicsLab 