# Machine Learning Hyperparameter Extraction

> Use when you have completed cross-validation tuning of one or more machine-learning models (e.g., AdaBoost, SVM, Random Forest) on a development dataset using the caret package and need to identify which specific hyperparameter values were selected as optimal.

- Skill: `holobiomicslab/machine-learning-hyperparameter-extraction` (Agent Skill)
- Install (CLI): `npx skillmds@latest add holobiomicslab/machine-learning-hyperparameter-extraction`
- Raw SKILL.md: https://api.skillmd.com/api/skills/holobiomicslab/machine-learning-hyperparameter-extraction/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- License: CC-BY-4.0
- Author: HolobiomicsLab (https://skillmd.com/u/holobiomicslab)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/holobiomicslab/machine-learning-hyperparameter-extraction

---


# Machine-Learning Hyperparameter Extraction

> **License: restricted** — no clear open-source license detected for the underlying tool; verify licensing before commercial use or redistribution. <!-- asb-license-banner -->
## Summary

Extract and validate optimized hyperparameter values (e.g., nIter, method) for trained machine-learning classifiers from cross-validation output. This skill identifies which hyperparameter combinations were selected during model tuning and documents them for reproducibility and model retraining.

## When to use

You have completed cross-validation tuning of one or more machine-learning models (e.g., AdaBoost, SVM, Random Forest) on a development dataset using the caret package and need to identify which specific hyperparameter values were selected as optimal. This is necessary before training a final classifier on the full development set or before applying the model to held-out test data.

## When NOT to use

- You are using pre-trained models from external sources without access to their cross-validation history or prediction data frames.
- Hyperparameter tuning was not performed; the model uses default hyperparameters and no runCrossValidation() output is available.
- The evaluation measures output is unavailable or does not align with the pred data frame (mismatched cross-validation runs).

## Inputs

- List object returned by runCrossValidation() containing trained models
- Prediction data frame from models$[AlgorithmName]$pred
- Evaluation measures data frame from getEvaluationMeasures()
- Cross-validation parameters (k-fold number, repetition number, metric set)

## Outputs

- Extracted hyperparameter combinations (e.g., nIter, method values)
- Identified optimized hyperparameter set for best-performing model configuration
- Validated correspondence between hyperparameters and evaluation metrics

## How to apply

After running runCrossValidation() with specified k-fold and repetition parameters (e.g., k=5, repNum=10) and a selected metric set (e.g., metricSet=c('M4','M7','M11')), access the prediction data frame returned by the trained model object (e.g., models$AdaBoost_M11$pred). Extract the unique hyperparameter columns (nIter, method, etc.) from this pred data frame to identify all candidate hyperparameter combinations tested. Cross-reference the extracted combinations against the evaluation measures output (generated by getEvaluationMeasures()) to confirm which hyperparameter set corresponds to the best-performing configuration according to your chosen evaluation metric. Document the optimized values for use in the subsequent trainClassifier() call on the full development set.

## Related tools

- **caret** (R package used to implement machine-learning algorithms and manage cross-validation hyperparameter tuning)
- **MetaClean** (Peak-quality classifier package providing wrapper functions runCrossValidation(), getEvaluationMeasures(), and trainClassifier() that manage hyperparameter tuning and extraction workflow) — https://github.com/KelseyChetnik/MetaClean
- **R** (Statistical computing environment in which hyperparameter extraction, data frame manipulation, and validation are performed)

## Examples

```
models <- runCrossValidation(trainData = pqm_development, k = 5, repNum = 10, rand.seed = 512, models = 'all', metricSet = c('M4','M7','M11')); hyperparams <- unique(models$AdaBoost_M11$pred[, c('nIter', 'method')]); eval_measures <- getEvaluationMeasures(models); best_idx <- which.max(eval_measures$Accuracy); optimal_hyperparams <- hyperparams[best_idx, ]
```

## Evaluation signals

- The extracted hyperparameter columns (nIter, method) are present and non-null in the pred data frame.
- The unique hyperparameter combinations extracted match the set of combinations that were iterated during cross-validation (no spurious or missing values).
- The optimized hyperparameter set identified can be cross-referenced to the row(s) in the evaluation measures output with the best (highest or lowest, depending on metric direction) performance score.
- The hyperparameter values are numeric or character strings matching the data types expected by trainClassifier() (e.g., nIter is an integer, method is a character string such as 'Adaboost.M1').
- When the optimized hyperparameters are passed to trainClassifier() on the full development set, model training succeeds without parameter-type or range errors.

## Limitations

- The article does not provide the hyperparameter values in the reported text; extraction relies entirely on programmatic access to the pred data frame, which is not published in the paper.
- Cross-validation outputs are sensitive to random seed and sampling strategy; reproducibility requires recording the exact runCrossValidation() parameters (k, repNum, rand.seed) used during tuning.
- If multiple hyperparameter combinations achieve similar evaluation scores, the extraction method does not disambiguate; practitioners must apply domain knowledge or additional criteria to select among tied candidates.
- The skill applies to MetaClean's wrapper functions and caret-compatible algorithms; extraction workflows may differ for other machine-learning frameworks or packages.

## Evidence

- [other] Load the trained models list returned by runCrossValidation() with parameters k=5, repNum=10, rand.seed=512, models='all', metricSet=c('M4','M7','M11') on pqm_development (500 peaks, 89 samples). 2. Access the prediction data frame from models$AdaBoost_M11$pred. 3. Extract the unique hyperparameter combinations (nIter and method columns) from the pred data frame.: "Access the prediction data frame from models$AdaBoost_M11$pred. 3. Extract the unique hyperparameter combinations (nIter and method columns) from the pred data frame."
- [other] Identify and record the optimized hyperparameters determined during cross-validation (nIter=150, method='Adaboost.M1' per the example). 5. Validate that these hyperparameters correspond to the best-performing AdaBoost configuration by cross-referencing with evaluation measures output.: "Validate that these hyperparameters correspond to the best-performing AdaBoost configuration by cross-referencing with evaluation measures output."
- [methods] The runCrossValidation function is a wrapper function that uses cross-validation to train a user-selected subset of the 8 available algorithms.: "The runCrossValidation function is a wrapper function that uses cross-validation to train a user-selected subset of the 8 available algorithms."
- [methods] We use the getEvaluationMeasures function to do this.: "We use the getEvaluationMeasures function to do this."
- [methods] metaclean_model <- trainClassifier(trainData = pqm_development, model = "AdaBoost", metricSet = "M11", hyperparameters = hyperparameters): "trainClassifier(trainData = pqm_development, model = "AdaBoost", metricSet = "M11", hyperparameters = hyperparameters)"