Rsf7 Benchmark Eval

rsf7-benchmark-eval

Maximum Likelihood Estimation based on Random Subspace EDA: Application to Extrasolar Planet Detection — Liu et al. (2017) (arXiv:1704.05761, 2017)

What this evaluates

Evaluates the optimization capability of evolutionary algorithms on a highly multimodal, nonseparable benchmark function across low (d=5) and high (d=20) dimensional settings. It measures how effectively the algorithm navigates complex, multi-peaked likelihood surfaces to locate the global optimum within a fixed computational budget.

Datasets

• Rotated Schaffers F7 (RSF7) — total ?; splits: test (-1)

Metrics

• mean_max_function_value (primary) — range: [0, 800]
  • Average of the maximum function value $g(\theta)$ found across 50 independent runs. The theoretical global maximum is 800.

Input / output format

Input: A parameter vector $\theta$ of dimension $d$ (5 or 20) to be evaluated by the objective function $g(\theta) = -f(\theta)$.

Output: The estimated maximum function value $g(\theta)$ achieved by the algorithm within the evaluation budget.

Scoring recipe

def score(predictions, gold):
    # predictions: list of max values from 50 runs
    # gold: 800 (theoretical max)
    mean_val = sum(predictions) / len(predictions)
    return mean_val

Common pitfalls

• The benchmark is originally a minimization function (RSF7) but is negated to $g(\theta)=-f(\theta)$ for maximization; the target value is 800, not 0.
• Evaluation budgets and population sizes scale significantly with dimension: $d=5$ uses $10^6$ evals and pop=20000, while $d=20$ uses $5\times10^6$ evals and pop=200000.
• Baseline comparisons rely on specific third-party implementations (e.g., CMA-ES diagonal option) with default parameters, making exact reproduction difficult without the cited code.

Evidence (verbatim from paper)

We ran each method for 50 times and calculated the mean of its estimate on the maximal function value. ... The maximum function value of $g(\theta)$ is 800, which is the target for the algorithm to search.

Citation

@misc{liu2017maximum,
  title={Maximum Likelihood Estimation based on Random Subspace EDA: Application to Extrasolar Planet Detection},
  author={Liu et al. (2017)},
  year={2017},
  note={arXiv:1704.05761}
}

• arXiv: 1704.05761