Rideshare Fairness Profit Eval

rideshare-fairness-profit-eval

Balancing the Tradeoff between Profit and Fairness in Rideshare Platforms During High-Demand Hours — Nanda et al. (2019) (arXiv:1912.08388, 2019)

What this evaluates

Evaluates online bipartite matching algorithms for rideshare platforms on their ability to balance total trip profit and group-level fairness (subgroup representation) during peak demand hours.

Datasets

• NYC Yellow Cabs 2013 — total 35109; splits: test (35109)
• Synthetic Rideshare — total 700; splits: test (700)

Metrics

• competitive ratio of profit (primary) — range: [0, 1]
  • The total profit achieved by the algorithm divided by the optimal profit obtained from a linear programming (LP) benchmark.
• competitive ratio of fairness — range: [0, 1]
  • The total fairness score achieved by the algorithm divided by the optimal fairness score obtained from a linear programming (LP) benchmark.

Input / output format

Input: Bipartite matching instance with driver types U, request types V, edge existence probabilities pf, edge profits wf, arrival rates, and cancellation quota Δ.

Output: Sequential driver-request assignments (matching) generated as requests arrive.

Scoring recipe

def evaluate_algorithm(algorithm, instances, n_runs=5000):
    total_profit = 0
    total_fairness = 0
    for _ in range(n_runs):
        profit, fairness = algorithm.run(instances)
        total_profit += profit
        total_fairness += fairness
    avg_profit = total_profit / n_runs
    avg_fairness = total_fairness / n_runs
    cr_profit = avg_profit / lp_optimal_profit
    cr_fairness = avg_fairness / lp_optimal_fairness
    return cr_profit, cr_fairness

Common pitfalls

• Race labels for drivers and riders are randomly assigned with fixed ratios (1:2 and 3:1) to simulate fairness constraints, not based on actual demographic data.
• The evaluation assumes each driver serves exactly one trip and is then removed, ignoring real-world multi-trip dynamics.
• Competitive ratios are computed against an LP benchmark that assumes full knowledge of request distributions, which may overestimate the difficulty of the problem.

Evidence (verbatim from paper)

We use profit computed via LP-([1]) as the benchmark and use it to calculate the competitive ratio of profit for NAdap. Similarly, we use LP-([2]) as the benchmark for fairness and use it to calculate the competitive ratio of fairness for NAdap. We run NAdap for 5000 iterations and take the average values over these runs to be the expectations.

Citation

@misc{nanda2019balancing,
  title={Balancing the Tradeoff between Profit and Fairness in Rideshare Platforms During High-Demand Hours},
  author={Nanda et al. (2019)},
  year={2019},
  note={arXiv:1912.08388}
}

• arXiv: 1912.08388