qhjqhj00 elliptic-aml-eval
Regulatory Graphs and GenAI for Real-Time Transaction Monitoring and Compliance Explanation in Banking — Khanvilkar et al. (2025) (arXiv:2506.01093, 2025)
What this evaluates
Evaluates a graph neural network's ability to classify Bitcoin transactions as licit or illicit using structural and narrative features, while testing a retrieval-augmented generation pipeline for producing regulatory-aligned explanations.
Datasets
- Elliptic AML dataset — total ?; splits: train (-1), test (-1)
Metrics
F1-score(primary) — range: percent- Harmonic mean of precision and recall: 2 * (precision * recall) / (precision + recall).
Input / output format
Input: Directed transaction edges between address nodes, augmented with structural features (in/out degree, betweenness centrality, transaction frequency) and synthetically generated narrative fields encoded into dense vectors via a fine-tuned DistilBERT model.
Output: Binary classification label (licit or illicit) per node. For suspicious transactions, a natural-language explanation generated by GPT-4 aligned with retrieved regulatory clauses.
Scoring recipe
def compute_f1(predictions, gold):
tp = sum(1 for p, g in zip(predictions, gold) if p == 1 and g == 1)
fp = sum(1 for p, g in zip(predictions, gold) if p == 1 and g == 0)
fn = sum(1 for p, g in zip(predictions, gold) if p == 0 and g == 1)
precision = tp / (tp + fp) if (tp + fp) > 0 else 0
recall = tp / (tp + fn) if (tp + fn) > 0 else 0
return 2 * (precision * recall) / (precision + recall) if (precision + recall) > 0 else 0
Common pitfalls
- Temporal split prevents random shuffling or standard cross-validation, making results sensitive to chronological data distribution shifts.
- Synthetic narrative fields may not capture the semantic complexity or noise of real-world financial memo descriptions.
- GPT-4 generation is non-deterministic, so explanation quality cannot be reliably reproduced without fixed seeds or temperature settings.
Evidence (verbatim from paper)
The dataset was divided into 80% training and 20% testing, maintaining chronological order to preserve the temporal integrity of the stream. Due to this time-dependent nature, no cross-validation was applied. The system achieved 98.2% F1-score and validated interpretability in expert evaluations.
Citation
@misc{khanvilkar2025regulatory,
title={Regulatory Graphs and GenAI for Real-Time Transaction Monitoring and Compliance Explanation in Banking},
author={Khanvilkar et al. (2025)},
year={2025},
note={arXiv:2506.01093}
}
- arXiv: 2506.01093