# Openfgl Eval

> Evaluates federated graph learning (FGL) algorithms across effectiveness, robustness, and efficiency dimensions. It probes how well distributed GNN training handles data heterogeneity, local noise/sparsity, low client participation, and privacy constraints across diverse graph topologies and simulation scenarios. Use when the user wants to benchmark on MUTAG, BZR, COX2, ENZYMES, DD, PROTEINS, COLLAB, BINARY, MULTI, Cora, CiteSeer, PubMed, Photo, Computers, Products, Chameleon, Actor, Ratings, DHFR, Squirrel, Questions, AIDS, NCI1, CS, Physics, or asks about evaluating this task. Reports test accuracy.

- Skill: `qhjqhj00/openfgl-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/openfgl-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/openfgl-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Productivity
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/openfgl-eval

---


# openfgl-eval

> OpenFGL: A Comprehensive Benchmark for Federated Graph Learning — Li et al. (2024) (arXiv:2408.16288, 2024)

## What this evaluates

Evaluates federated graph learning (FGL) algorithms across effectiveness, robustness, and efficiency dimensions. It probes how well distributed GNN training handles data heterogeneity, local noise/sparsity, low client participation, and privacy constraints across diverse graph topologies and simulation scenarios.

## Datasets

- **MUTAG** — total ?; splits: test (-1)
- **BZR** — total ?; splits: test (-1)
- **COX2** — total ?; splits: test (-1)
- **ENZYMES** — total ?; splits: test (-1)
- **DD** — total ?; splits: test (-1)
- **PROTEINS** — total ?; splits: test (-1)
- **COLLAB** — total ?; splits: test (-1)
- **BINARY** — total ?; splits: test (-1)
- **MULTI** — total ?; splits: test (-1)
- **Cora** — total ?; splits: test (-1)
- **CiteSeer** — total ?; splits: test (-1)
- **PubMed** — total ?; splits: test (-1)
- **Photo** — total ?; splits: test (-1)
- **Computers** — total ?; splits: test (-1)
- **Products** — total ?; splits: test (-1)
- **Chameleon** — total ?; splits: test (-1)
- **Actor** — total ?; splits: test (-1)
- **Ratings** — total ?; splits: test (-1)
- **DHFR** — total ?; splits: test (-1)
- **Squirrel** — total ?; splits: test (-1)
- **Questions** — total ?; splits: test (-1)
- **AIDS** — total ?; splits: test (-1)
- **NCI1** — total ?; splits: test (-1)
- **CS** — total ?; splits: test (-1)
- **Physics** — total ?; splits: test (-1)

## Metrics

- `test accuracy` **(primary)** — range: percent
  - Percentage of correctly predicted node labels on the held-out test set, calculated as (number of correct predictions / total test samples) * 100.

## Input / output format

**Input**: Graph-structured data (node features, adjacency matrices/edges, and labels) partitioned across multiple clients according to federated simulation strategies (Graph-FL or Subgraph-FL), with optional injected noise, sparsity, or privacy constraints.

**Output**: Predicted node labels for each test node, aggregated into a test accuracy percentage per dataset and algorithm configuration.

## Scoring recipe

```python
def compute_test_accuracy(predictions, gold_labels):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    return (correct / len(gold_labels)) * 100
```

## Common pitfalls

- Federated collaboration yields diminishing returns on small-scale datasets (e.g., MUTAG, BZR, COX2) due to insufficient data to support effective multi-client training.
- Topology-driven FGL algorithms are highly sensitive to edge noise, which severely disrupts local updates and server-side collaboration more than feature or label noise.
- Some baselines fail on large-scale datasets (e.g., ogbn-products) due to out-of-memory (OOM) errors, limiting fair scalability comparisons in real-world deployments.

## Evidence (verbatim from paper)

> To answer Q1, in addition to the federated multi-client collaboration, we introduce "Local" to represent solely local training for analyzing the advantages and potential limitations of FGL. Based on this, to answer Q2, we present the end-to-end performance in Table 6 and Table 7. Table 6: Graph-FL test accuracy (%) . The best result is bold.

## Citation

```bibtex
@misc{li2024openfgl,
  title={OpenFGL: A Comprehensive Benchmark for Federated Graph Learning},
  author={Li et al. (2024)},
  year={2024},
  note={arXiv:2408.16288}
}
```

- arXiv: 2408.16288