Svc Ongoing Eval

svc-ongoing-eval

SVC-onGoing: Signature Verification Competition — Tolosana et al. (2021) (arXiv:2108.06090, 2021)

What this evaluates

Evaluates on-line signature verification systems across office (stylus), mobile (finger), and hybrid scenarios. It measures robustness against both skilled and random forgeries, testing generalization across different acquisition devices and intra-user variability.

Datasets

• DeepSignDB — total 442; splits: eval (442)
• SVC2021_EvalDB — total ?; splits: test (-1); repo https://github.com/BiDAlab/SVC2021_EvalDB

Metrics

• EER (primary) — range: percent
  • Equal Error Rate, defined as the operating point where the False Acceptance Rate (FAR) equals the False Rejection Rate (FRR). Reported as a percentage.

Input / output format

Input: On-line signature traces comprising X and Y spatial coordinates, time, and pressure signals.

Output: Binary verification decision (genuine vs. forgery) or a continuous similarity score used to compute FAR/FRR curves.

Scoring recipe

def compute_eer(scores, labels):
    # scores: similarity scores, labels: 1 (genuine), 0 (forgery)
    thresholds = np.unique(scores)
    best_eer = 1.0
    for t in thresholds:
        far = np.mean(scores[labels == 0] >= t)
        frr = np.mean(scores[labels == 1] < t)
        eer = max(far, frr)
        if eer < best_eer:
            best_eer = eer
    return best_eer * 100

Common pitfalls

• Models trained exclusively on stylus data often fail to generalize to finger-written mobile signatures due to high intra-user variability and device-specific acquisition noise.
• Evaluating systems on only one forgery type (skilled or random) is insufficient; the protocol requires simultaneous robustness against both to simulate real-world conditions.
• Performance on the development set (DeepSignDB) frequently overestimates final evaluation results (SVC2021_EvalDB) due to distribution shifts in subjects and acquisition devices.

Evidence (verbatim from paper)

Concretely, for each of the tasks, the DLVC-Lab team achieves relative improvements of 74.54%, 50.34%, and 58.3% EER compared to the Baseline DTW. These results prove the high potential of deep learning approaches such as DSDTW and TA-RNN for the on-line signature verification field, as commented in previous studies.

Citation

@misc{tolosana2021svcongoing,
  title={SVC-onGoing: Signature Verification Competition},
  author={Tolosana et al. (2021)},
  year={2021},
  note={arXiv:2108.06090}
}

• arXiv: 2108.06090