Wds Pareto Front Eval

wds-pareto-front-eval

Water Distribution System Design Using Multi-Objective Particle Swarm Optimisation — Patil et al. (2019) (arXiv:1903.06127, 2019)

What this evaluates

Evaluates the quality and diversity of multi-objective optimization algorithms for water distribution system design by comparing generated Pareto fronts against established benchmark fronts. It measures coverage of known solutions, discovery of novel non-dominated designs, and computational efficiency.

Datasets

• HAN, NYT, BLA, and GOY networks — total ?; splits: test (-1)

Metrics

• N_A^u, N_B^u, N_A^a, N_B^a, N_c, N_{FE} (primary) — range: count
  • Counts comparing two Pareto fronts (PF-A and PF-B): N_A^u (solutions in PF-A not covered by PF-B), N_B^u (new solutions in PF-B not in PF-A), N_A^a/N_B^a (covered solutions), N_c (common solutions), and N_{FE} (total function evaluations). Solutions are compared for dominance and exact coverage in objective space.

Input / output format

Input: Water distribution network topology, pipe/nodes data, demand requirements, and cost/resilience objective functions.

Output: A set of non-dominated design configurations (e.g., pipe diameters, pump settings) representing the Pareto front.

Scoring recipe

def score_pf(pf_a, pf_b, n_fe):
    covered_a, uncovered_a, dominated_a = 0, 0, 0
    covered_b, uncovered_b, dominated_b = 0, 0, 0
    common = 0
    for sol_a in pf_a:
        if any(sol_b == sol_a for sol_b in pf_b):
            covered_a += 1; common += 1
        elif any(sol_b dominates sol_a for sol_b in pf_b):
            dominated_a += 1
        else:
            uncovered_a += 1
    # Repeat for pf_b against pf_a
    return {'N_A^u': uncovered_a, 'N_B^u': uncovered_b, 'N_A^a': covered_a, 'N_B^a': covered_b, 'N_c': common, 'N_{FE}': n_fe}

Common pitfalls

• Evaluating only single-objective metrics like average or minimum cost, which ignores trade-offs and diversity in the Pareto front.
• Focusing solely on objective-space proximity while ignoring decision-space diversity, as solutions can be nearly identical in cost/resilience but vastly different in physical design.
• Comparing front quality without normalizing for computational budget (function evaluations), leading to unfair advantages for more expensive runs.

Evidence (verbatim from paper)

Table 3 shows the results (i.e., $N_A^t$, $N_A^u$, etc. as defined in Sec. 2) for the HAN network... In each case, a substantial number of new ND solutions ($N_B^u$) have been found by MOPSO+. Furthermore, almost all ND solutions in the UExeter PFs are covered by MOPSO+ (see the $N_A^u$ column in the table). The metrics proposed in Sec. 2 therefore seem to be more attractive from a practical perspective than metrics such as average cost, lowest cost, standard deviation, or spread.

Citation

@misc{patil2019water,
  title={Water Distribution System Design Using Multi-Objective Particle Swarm Optimisation},
  author={Patil et al. (2019)},
  year={2019},
  note={arXiv:1903.06127}
}

• arXiv: 1903.06127