Ca Afp Eval

ca-afp-eval

CA-AFP: Cluster-Aware Adaptive Federated Pruning — Jha et al. (2026) (arXiv:2603.01739, 2026)

What this evaluates

Evaluates federated learning models for human activity recognition under non-IID data distributions. It measures classification accuracy, fairness across heterogeneous clients, and communication efficiency during model pruning and clustering.

Datasets

• WISDM — total 10591; splits: test (-1)
• UCI-HAR — total 10299; splits: test (-1)

Metrics

• Accuracy (primary) — range: percent
  • Mean test accuracy across all federated clients.
• Fairness — range: percent
  • Standard deviation of per-client test accuracies. Lower values indicate more balanced performance across clients.
• Communication Cost — range: MB
  • Total volume of data transmitted between server and clients over R rounds. Calculated as sum over rounds and selected clients of 2 * |W| * (1 - sparsity) * 4 / 1024^2, accounting for bidirectional transfers.

Input / output format

Input: Multivariate time-series sensor data (accelerometer/gyroscope) segmented into fixed-length windows. Shape is (200, 3) for WISDM and (128, 9) for UCI-HAR.

Output: Softmax probability distribution over 6 activity classes (Walking, Jogging, Upstairs, Downstairs, Sitting, Standing).

Scoring recipe

def compute_metrics(per_client_accuracies, rounds, selected_clients, num_params, sparsities):
    accuracy = np.mean(per_client_accuracies)
    fairness = np.std(per_client_accuracies)
    comm_cost = 0.0
    for r in range(rounds):
        for k in selected_clients[r]:
            comm_cost += 2 * num_params * (1 - sparsities[k][r]) * 4 / (1024**2)
    return accuracy, fairness, comm_cost

Common pitfalls

• Fairness is defined as the standard deviation of client accuracies (lower is better), not a ratio or inequality index.
• Communication cost explicitly includes a factor of 2 to account for bidirectional (client-to-server and server-to-client) parameter transfers.
• Sparsity is fixed at ~70% for all compared methods in the main results, so accuracy drops are primarily due to pruning-induced regularization rather than varying sparsity levels.

Evidence (verbatim from paper)

To compare with the existing baselines, and to analyse ablations, we use three metrics, namely Accuracy, Fairness (Standard Deviation) and Communication Cost (in MB). The average Accuracy is the mean test accuracy ($\mu$) across all clients. We use the term ‘fairness’ to refer to the standard deviation ($\sigma$) of the mean accuracy across all clients.

Citation

@misc{jha2026caafp,
  title={CA-AFP: Cluster-Aware Adaptive Federated Pruning},
  author={Jha et al. (2026)},
  year={2026},
  note={arXiv:2603.01739}
}

• arXiv: 2603.01739