RAG Har Eval

rag-har-eval

RAG-HAR: Retrieval Augmented Generation-based Human Activity Recognition — Sivaroopan et al. (2025) (arXiv:2512.08984, 2025)

What this evaluates

Evaluates a training-free, retrieval-augmented framework for classifying human activities from wearable sensor time-series data. It probes the model's ability to perform open-world activity recognition by retrieving semantically similar sensor examples and using an LLM to predict activity labels without fine-tuning.

Datasets

• HHAR — total ?; splits: test (-1)
• PAMAP2 — total ?; splits: test (-1)
• MHEALTH — total ?; splits: test (-1)
• GOTOV — total ?; splits: test (-1)
• SKODA — total ?; splits: test (-1)
• USC-HAD — total ?; splits: test (-1)

Metrics

• accuracy (primary) — range: [0, 1]
  • Standard classification accuracy: the number of correctly predicted activity labels divided by the total number of test instances.

Input / output format

Input: Sliding-window segmented sensor time-series data (accelerometer/gyroscope/IMU channels). The pipeline first converts windows to statistical descriptors, retrieves top-k similar examples, and feeds the query window plus retrieved contexts to an LLM.

Output: Predicted activity class label.

Scoring recipe

correct = 0
total = 0
for pred, gold in zip(predictions, gold_labels):
    if pred == gold:
        correct += 1
    total += 1
return correct / total

Common pitfalls

• Sliding window sizes and step sizes vary significantly across datasets (e.g., 2s vs 4s windows, different overlap percentages), requiring dataset-specific preprocessing.
• Data preparation scripts and indexing procedures are delegated to external citations, making exact replication difficult without accessing those references.
• Retrieval configuration (q=10, weighted re-ranking) is fixed; changing these hyperparameters may alter accuracy without changing the core protocol.

Evidence (verbatim from paper)

We generated embeddings using text-embedding-3-small [openaiIntroducingEmbedding], as this model consistently provided more stable retrieval performance compared to alternatives. These embeddings (dimension = 1536) were indexed in Zilliz [zilliz], a high-performance cloud vector database, which natively supports the embedding output size. During retrieval, we applied weighted re-ranking with weights (0.4, 0.2, 0.2, 0.2). Among the segmentations tested, the [full, start, mid, end] configuration yielded the best balance of coverage and accuracy. We fixed the number of retrieved contexts ($q$) to 10, as higher values increased inference cost without accuracy gains, while lower values reduced robustness. For the classification stage, we employed gpt-5-mini [openaiIntroducingGpt5], which offered the best trade-off between accuracy and cost after experiments with other models (openai, gemini and llama variants). Finally, for prompt optimization, we used gpt-5 [openaiIntroducingGpt5], leveraging its stronger reasoning for more effective refinements.

Citation

@misc{sivaroopan2025raghar,
  title={RAG-HAR: Retrieval Augmented Generation-based Human Activity Recognition},
  author={Sivaroopan et al. (2025)},
  year={2025},
  note={arXiv:2512.08984}
}

• arXiv: 2512.08984