# Regression Model Evaluation Metrics

> Use when after training a deep-learning regression model (e.g., for CCS prediction from voxel projected area features), evaluate it on held-out test data to report per-molecule predictions and aggregate performance metrics.

- Skill: `holobiomicslab/regression-model-evaluation-metrics` (Agent Skill)
- Install (CLI): `npx skillmds@latest add holobiomicslab/regression-model-evaluation-metrics`
- Raw SKILL.md: https://api.skillmd.com/api/skills/holobiomicslab/regression-model-evaluation-metrics/raw
- Safety review: PASS (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- License: CC-BY-4.0
- Author: HolobiomicsLab (https://skillmd.com/u/holobiomicslab)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/holobiomicslab/regression-model-evaluation-metrics

---


# regression-model-evaluation-metrics

## Summary

Compute and interpret performance metrics on held-out test data for continuous-value regression models, such as CCS prediction from molecular features. This skill ensures trained models generalize beyond training and validation sets by quantifying prediction accuracy across multiple statistical measures.

## When to use

After training a deep-learning regression model (e.g., for CCS prediction from voxel projected area features), evaluate it on held-out test data to report per-molecule predictions and aggregate performance metrics. Apply this skill whenever you need to validate model accuracy on unseen molecules and generate publishable performance reports.

## When NOT to use

- Input is already a feature table without ground-truth labels — metrics require paired predictions and reference values.
- Test data was not held out during training or hyperparameter tuning — evaluation will be overly optimistic and not reflect generalization.
- Model output is categorical or probabilistic rather than continuous — use classification metrics (accuracy, F1, AUC) instead.

## Inputs

- trained PyTorch neural network model (checkpoint file)
- test dataset (SMILES, adduct type, ground-truth CCS labels)
- computed test features (voxel projected area, molecular graph, m/z, adduct encoding)

## Outputs

- per-molecule CCS predictions (numerical values)
- regression performance metrics (MAE, RMSE, R², MRE)
- model performance summary report (aggregated statistics)

## How to apply

Load the trained PyTorch model and the held-out test dataset (containing molecular features: voxel projected area, molecular graph, adduct one-hot encoding, m/z, and ground-truth CCS labels). Feed test features through the model to generate per-molecule CCS predictions. Compute standard regression metrics: mean absolute error (MAE), root mean squared error (RMSE), coefficient of determination (R²), and optionally mean relative error (MRE). Report predictions alongside metrics to assess both individual prediction quality and overall model generalization. Inspect residuals (predicted − actual) for systematic bias or heteroscedasticity.

## Related tools

- **PyTorch** (Load trained regression model checkpoint and compute batch predictions on test data) — https://pytorch.org/
- **pandas** (Organize test features, ground-truth labels, and predicted CCS values into DataFrames for metric computation and export) — https://pandas.pydata.org/
- **NumPy** (Compute regression metrics (MAE, RMSE, R²) and residual analysis via vectorized operations) — https://numpy.org/
- **scikit-learn** (Provide standard regression metrics functions (mean_absolute_error, mean_squared_error, r2_score))

## Examples

```
from PACCS.Prediction import PACCS_predict; PACCS_predict(input_path='data/external_test_set.csv', model_path='trained_model.pth', output_path='test_predictions.csv')
```

## Evaluation signals

- R² value is positive and close to 1 (>0.8 typical for well-fit models); negative R² indicates model performs worse than a simple mean baseline
- MAE and RMSE are in physically plausible units (Ångströms² for CCS); compare against literature benchmarks and external test set performance
- Per-molecule prediction residuals (actual − predicted) are normally distributed with mean near zero; systematic bias or heteroscedasticity suggests model deficiency
- Test set metrics are comparable to validation metrics from training; large gap suggests overfitting
- External test set predictions (if available) show similar accuracy to curated dataset test fold, confirming generalization

## Limitations

- Metrics are computed only on the held-out test set; performance on new, real-world molecules may differ if they have different chemical distributions.
- Single-fold evaluation does not capture variance across random seeds or data splits; k-fold or repeated-hold-out evaluation is more robust.
- RMSE is sensitive to outlier predictions; MAE may be more interpretable for applications with extreme values.
- R² can be misleading for non-linear relationships or when prediction errors are systematically biased in certain input regions.
- README mentions 8:1:1 split (training:validation:test) but does not specify random seed or stratification strategy, affecting reproducibility.

## Evidence

- [other] Evaluate the trained model on held-out test data and output per-molecule CCS predictions alongside model performance metrics.: "Evaluate the trained model on held-out test data and output per-molecule CCS predictions alongside model performance metrics."
- [readme] The curated dataset is randomly split into the training, validation, and test sets in a ratio of 8:1:1.: "The curated dataset is randomly split into the training, validation, and test sets in a ratio of 8:1:1."
- [other] Train the model on the prepared dataset using PyTorch, optimizing for CCS prediction accuracy across epochs with appropriate loss function and learning rate scheduling.: "Train the model on the prepared dataset using PyTorch, optimizing for CCS prediction accuracy across epochs"
- [readme] The predicted CCS values of molecules are obtained by feeding the voxel projected area, molecular graph, one-hot encoding of adduct type, and m/z into the already trained PACCS model: "The predicted CCS values of molecules are obtained by feeding the voxel projected area, molecular graph, one-hot encoding of adduct type, and m/z into the already trained PACCS model"