missing-value-imputation-with-k-nearest-neighbour
Summary
Apply k-nearest neighbour imputation to fill missing values in metabolomics assays after threshold-based filtering and batch intersection. This skill restores metabolite quantification across multi-batch studies by leveraging local similarity structure in the feature space.
When to use
You have log-transformed metabolomics data in SummarizedExperiment format organized as multiple batches, have already filtered metabolites with >50% missing values per batch and retained only those quantified across all batches via intersect method, and now need to impute remaining missing values before downstream normalisation or statistical analysis.
When NOT to use
- Input metabolomics data is already complete (no missing values) — imputation adds computational overhead without benefit.
- Missing values are >50% per batch in most metabolites — threshold filtering would eliminate too many features, leaving insufficient data for k-nearest neighbour to work reliably.
- Data is not yet log-transformed or organized as SummarizedExperiment objects — the skill expects logRaw assay as input.
Inputs
- List of SummarizedExperiment objects, each with a logRaw assay containing log2-transformed metabolite raw counts and sample-level missing-value patterns
- Metabolomics data organized as multiple experimental batches with samples and metabolite features
Outputs
- List of SummarizedExperiment objects with a new rawImpute assay containing imputed metabolite quantification with no missing values
- Reduced feature set: metabolites quantified across all batches with k-nearest neighbour-imputed values
How to apply
Apply hRUV::clean() with threshold=0.5, method='intersect', assay='logRaw', and newAssay='rawImpute' to a list of SummarizedExperiment objects. The function first filters metabolites with >50% missing values per batch, retains only metabolites present in all batches using the intersect method, then performs k-nearest neighbour imputation on the filtered feature set. The resulting rawImpute assay contains complete metabolite quantification with no missing values across all samples and batches. Verify success by confirming the rawImpute assay has reduced metabolite count (relative to input) and zero missing values.
Related tools
- hRUV (Primary tool: implements clean() function for threshold filtering, batch intersection, and k-nearest neighbour imputation on metabolomics batches) — https://github.com/SydneyBioX/hRUV
- SummarizedExperiment (Data structure: stores assays (logRaw, rawImpute), sample metadata, and feature annotations for multi-batch metabolomics objects)
- DMwR2 (Dependency of hRUV: provides k-nearest neighbour imputation algorithm implementation)
- dplyr (Utility: data manipulation and pipeline composition in hRUV workflows)
Examples
dat_list = hRUV::clean(dat_list, threshold = 0.5, method = "intersect", assay = "logRaw", newAssay = "rawImpute")
Evaluation signals
- The rawImpute assay has reduced metabolite count compared to logRaw (e.g., 61 metabolites retained from larger initial set), reflecting successful threshold and intersect filtering.
- Zero missing values are present in the rawImpute assay across all samples and batches.
- Metabolites retained are exactly those quantified in all five batches (intersection rule respected).
- Imputed values are numeric and within the realistic range of log2-transformed metabolite abundance (no extreme outliers or NaN/Inf).
- Assay dimensions match: metabolites × samples are consistent between logRaw (pre-filtered) and rawImpute (post-filtered and imputed).
Limitations
- k-nearest neighbour imputation relies on local similarity structure; sparse or highly imbalanced feature spaces may yield unreliable imputations.
- The intersect method removes all metabolites not quantified in every batch, which can be overly stringent for studies with batch-specific biological signals.
- Missing-not-at-random (MNAR) mechanisms — e.g., values below instrument detection limits — may violate the missing-completely-at-random (MCAR) assumption implicit in k-nearest neighbour imputation.
- No guidance provided in the article on choosing k or on validation of imputation quality; practitioners should cross-validate against held-out complete cases if possible.
Evidence
- [intro] The
clean function filters metabolites with >50% missing values per batch using the intersect method, then performs k-nearest neighbour imputation to produce a rawImpute assay containing 61 metabolites quantified across all five batches.: "The clean function filters metabolites with >50% missing values per batch using the intersect method, then performs k-nearest neighbour imputation to produce a rawImpute assay containing 61"
- [intro] Apply hRUV::clean() with threshold=0.5, method='intersect', assay='logRaw', and newAssay='rawImpute' parameters on a list of SummarizedExperiment objects.: "dat_list = hRUV::clean(dat_list, threshold = 0.5, method = "intersect", assay = "logRaw", newAssay = "rawImpute")"
- [intro] We have filtered metabolites with more than 50% of missing values per batch and selected metabolites that are quantified across all batches (intersect): "We have filtered metabolites with more than 50% of missing values per batch and selected metabolites that are quantified across all batches (intersect)"
1---2name: missing-value-imputation-with-k-nearest-neighbour3description: Use when you have log-transformed metabolomics data in SummarizedExperiment format organized as multiple batches, have already filtered metabolites with >50% missing values per batch and retained only those quantified across all batches via intersect method, and now need to impute remaining missing.4license: CC-BY-4.05---67# missing-value-imputation-with-k-nearest-neighbour89## Summary1011Apply k-nearest neighbour imputation to fill missing values in metabolomics assays after threshold-based filtering and batch intersection. This skill restores metabolite quantification across multi-batch studies by leveraging local similarity structure in the feature space.1213## When to use1415You have log-transformed metabolomics data in SummarizedExperiment format organized as multiple batches, have already filtered metabolites with >50% missing values per batch and retained only those quantified across all batches via intersect method, and now need to impute remaining missing values before downstream normalisation or statistical analysis.1617## When NOT to use1819- Input metabolomics data is already complete (no missing values) — imputation adds computational overhead without benefit.20- Missing values are >50% per batch in most metabolites — threshold filtering would eliminate too many features, leaving insufficient data for k-nearest neighbour to work reliably.21- Data is not yet log-transformed or organized as SummarizedExperiment objects — the skill expects logRaw assay as input.2223## Inputs2425- List of SummarizedExperiment objects, each with a logRaw assay containing log2-transformed metabolite raw counts and sample-level missing-value patterns26- Metabolomics data organized as multiple experimental batches with samples and metabolite features2728## Outputs2930- List of SummarizedExperiment objects with a new rawImpute assay containing imputed metabolite quantification with no missing values31- Reduced feature set: metabolites quantified across all batches with k-nearest neighbour-imputed values3233## How to apply3435Apply hRUV::clean() with threshold=0.5, method='intersect', assay='logRaw', and newAssay='rawImpute' to a list of SummarizedExperiment objects. The function first filters metabolites with >50% missing values per batch, retains only metabolites present in all batches using the intersect method, then performs k-nearest neighbour imputation on the filtered feature set. The resulting rawImpute assay contains complete metabolite quantification with no missing values across all samples and batches. Verify success by confirming the rawImpute assay has reduced metabolite count (relative to input) and zero missing values.3637## Related tools3839- **hRUV** (Primary tool: implements clean() function for threshold filtering, batch intersection, and k-nearest neighbour imputation on metabolomics batches) — https://github.com/SydneyBioX/hRUV40- **SummarizedExperiment** (Data structure: stores assays (logRaw, rawImpute), sample metadata, and feature annotations for multi-batch metabolomics objects)41- **DMwR2** (Dependency of hRUV: provides k-nearest neighbour imputation algorithm implementation)42- **dplyr** (Utility: data manipulation and pipeline composition in hRUV workflows)4344## Examples4546```47dat_list = hRUV::clean(dat_list, threshold = 0.5, method = "intersect", assay = "logRaw", newAssay = "rawImpute")48```4950## Evaluation signals5152- The rawImpute assay has reduced metabolite count compared to logRaw (e.g., 61 metabolites retained from larger initial set), reflecting successful threshold and intersect filtering.53- Zero missing values are present in the rawImpute assay across all samples and batches.54- Metabolites retained are exactly those quantified in all five batches (intersection rule respected).55- Imputed values are numeric and within the realistic range of log2-transformed metabolite abundance (no extreme outliers or NaN/Inf).56- Assay dimensions match: metabolites × samples are consistent between logRaw (pre-filtered) and rawImpute (post-filtered and imputed).5758## Limitations5960- k-nearest neighbour imputation relies on local similarity structure; sparse or highly imbalanced feature spaces may yield unreliable imputations.61- The intersect method removes all metabolites not quantified in every batch, which can be overly stringent for studies with batch-specific biological signals.62- Missing-not-at-random (MNAR) mechanisms — e.g., values below instrument detection limits — may violate the missing-completely-at-random (MCAR) assumption implicit in k-nearest neighbour imputation.63- No guidance provided in the article on choosing k or on validation of imputation quality; practitioners should cross-validate against held-out complete cases if possible.6465## Evidence6667- [intro] The `clean` function filters metabolites with >50% missing values per batch using the intersect method, then performs k-nearest neighbour imputation to produce a rawImpute assay containing 61 metabolites quantified across all five batches.: "The `clean` function filters metabolites with >50% missing values per batch using the intersect method, then performs k-nearest neighbour imputation to produce a rawImpute assay containing 61"68- [intro] Apply hRUV::clean() with threshold=0.5, method='intersect', assay='logRaw', and newAssay='rawImpute' parameters on a list of SummarizedExperiment objects.: "dat_list = hRUV::clean(dat_list, threshold = 0.5, method = "intersect", assay = "logRaw", newAssay = "rawImpute")"69- [intro] We have filtered metabolites with more than 50% of missing values per batch and selected metabolites that are quantified across all batches (intersect): "We have filtered metabolites with more than 50% of missing values per batch and selected metabolites that are quantified across all batches (intersect)"
