K Barrier Prediction Eval

k-barrier-prediction-eval

A deep learning approach to predict the number of k-barriers for intrusion detection over a circular region using wireless sensor networks — Abhilash Singh et al. (2022) (arXiv:2208.11887, 2022)

What this evaluates

Evaluates a model's ability to predict the number of k-barriers for intrusion detection in wireless sensor networks deployed over circular regions, based on geometric and deployment parameters.

Datasets

• WSN k-barrier simulation dataset — total ?; splits: train (-1), val (-1), test (-1)

Metrics

• R — range: [-1, 1]
  • Pearson correlation coefficient between predicted and observed barrier counts. Higher values indicate better alignment with observed values.
• RMSE (primary) — range: other
  • Root Mean Squared Error: sqrt(mean((y_pred - y_true)^2)). Lower values indicate higher accuracy.
• bias — range: other
  • Mean prediction error: mean(y_pred - y_true). Positive values indicate overestimation, negative values indicate underestimation.

Input / output format

Input: Four numerical features: area of the circular region, sensor sensing range, transmission range, and total sensor count.

Output: A single continuous float representing the predicted number of k-barriers.

Scoring recipe

import numpy as np
from scipy.stats import pearsonr
def compute_metrics(y_true, y_pred):
    rmse = np.sqrt(np.mean((y_pred - y_true) ** 2))
    bias = np.mean(y_pred - y_true)
    r, _ = pearsonr(y_true, y_pred)
    return {'R': r, 'RMSE': rmse, 'bias': bias}

Common pitfalls

• The paper evaluates performance on training, validation, test, and combined datasets separately; reporting only combined accuracy masks generalization performance.
• Bias sign convention is explicitly defined: positive bias means overestimation, negative means underestimation, which is opposite to some ML frameworks that define bias as y_true - y_pred.
• Error distribution is noted to be slightly right-skewed, meaning mean bias may not fully capture model performance compared to median error or RMSE.

Evidence (verbatim from paper)

We have used R, RMSE, and bias as the performance metrics. A high value of R represents that the predicted values are well in accord with the observed value. A low value of RMSE represents a more accurate model. A positive value of bias shows overestimation, and a negative value of bias shows underestimation.

Citation

@misc{singh2022deep,
  title={A deep learning approach to predict the number of k-barriers for intrusion detection over a circular region using wireless sensor networks},
  author={Abhilash Singh et al. (2022)},
  year={2022},
  note={arXiv:2208.11887}
}

• arXiv: 2208.11887