Wivi Har Eval

wivi-har-eval

A Wireless-Vision Dataset for Privacy Preserving Human Activity Recognition — Hao et al. (2022) (arXiv:2205.11962, 2022)

What this evaluates

Evaluates human activity recognition (HAR) performance using only wireless Channel State Information (CSI) signals under varying action segmentation windows (1s, 2s, 3s). It probes the robustness of classification models in privacy-preserving environments where visual data is occluded or unavailable during testing.

Datasets

• WiVi — total ?; splits: train (-1), test (-1)

Metrics

• OA (primary) — range: [0, 1]
  • Overall Accuracy (OA) is the ratio of correctly classified instances to the total number of test instances.
• PA — range: [0, 1]
  • Per-action accuracy (referred to as product accuracy in the paper) is the ratio of correctly classified instances for a specific action class to the total instances of that class.

Input / output format

Input: Segmented Channel State Information (CSI) sequences (1s, 2s, or 3s windows), preprocessed and upsampled to 6x224x224 tensors for deep models, or raw/filtered CSI features for SVM.

Output: Discrete class label from 9 predefined human activities: falling, throwing, pushing, kicking, punching, jumping, phonetalk, seating, drinking.

Scoring recipe

def compute_metrics(predictions, labels):
    correct = sum(p == l for p, l in zip(predictions, labels))
    oa = correct / len(labels)
    pa = {}
    for action in actions:
        mask = [l == action for l in labels]
        if sum(mask) > 0:
            pa[action] = sum(p == l for p, l in zip(predictions, labels) if l == action) / sum(mask)
    return oa, pa

Common pitfalls

• The paper uses the non-standard term 'product accuracy' for PA, which actually refers to per-class accuracy.
• Evaluation is strictly performed on WiFi-only signals; vision data is only used for skeleton supervision during training, not during testing.
• Performance varies significantly by segmentation window (1s, 2s, 3s), so results must be reported per window rather than aggregated.

Evidence (verbatim from paper)

The dataset was split into a training and a testing sets with an 70 - 30% split respectively. All the test results on our dataset use only WiFi signals without vision-based input. As illustrated in Table.II, PA is product accuracy and OA represents overall accuracy.

Citation

@misc{hao2022wivi,
  title={A Wireless-Vision Dataset for Privacy Preserving Human Activity Recognition},
  author={Hao et al. (2022)},
  year={2022},
  note={arXiv:2205.11962}
}

• arXiv: 2205.11962