Sddfcs Simulation Eval

sddfcs-simulation-eval

Same-Day Delivery with Fairness — Xinwei Chen et al. (2020) (arXiv:2007.09541, 2020)

What this evaluates

Evaluates a reinforcement learning policy for same-day delivery routing on synthetic geographic settings, measuring the trade-off between overall service utility and regional fairness (minimum service rate).

Datasets

• SDDFCS Simulation — total 2000; splits: train (1500), test (500)

Metrics

• r_total (primary) — range: [0, 1]
  • Overall service rate calculated as the total number of accepted requests divided by the total number of requests across all regions for a given day.
• r_min — range: [0, 1]
  • Minimum regional service rate, representing the lowest acceptance rate among all regions for a given day, used as the fairness metric.

Input / output format

Input: A daily sequence of customer requests generated via a Poisson process, each with an arrival time, (x,y) coordinates, and assigned geographic region. Includes depot location and fleet size (3 or 5 vehicles).

Output: For each incoming request, the model outputs an acceptance decision (accept or reject) and, if accepted, the index of the vehicle assigned to serve it.

Scoring recipe

def compute_metrics(accepted_set, all_requests, region_map):
    total_rate = len(accepted_set) / len(all_requests)
    region_total = {}
    region_acc = {}
    for req in all_requests:
        r = region_map[req]
        region_total[r] = region_total.get(r, 0) + 1
    for req in accepted_set:
        r = region_map[req]
        region_acc[r] = region_acc.get(r, 0) + 1
    rates = {r: region_acc.get(r, 0) / region_total[r] for r in region_total}
    return total_rate, min(rates.values()), max(rates.values())

Common pitfalls

• Uses synthetic Poisson-generated requests and normal-distribution coordinates rather than real-world GPS data.
• Fairness metric (r_min) can be disproportionately affected by low request volumes in underserved regions.
• Results are averaged across four geography-fleet combinations, which may obscure setting-specific performance variations.

Evidence (verbatim from paper)

For each geography-fleet combination, we evaluate the model on the test set and calculate the utility as well as the acceptance rates of all regions. ... the vertical axis represents the service rates r_total (utility), r_min (fairness), and r_max.

Citation

@misc{chen2020samedeliveryfairness,
  title={Same-Day Delivery with Fairness},
  author={Xinwei Chen et al. (2020)},
  year={2020},
  note={arXiv:2007.09541}
}

• arXiv: 2007.09541