Soberdse Eval

soberdse-eval

SoberDSE: Sample-Efficient Design Space Exploration via Learning-Based Algorithm Selection — Lei Xu et al. (2026) (arXiv:2603.00986, 2026)

What this evaluates

Evaluates a learning-based algorithm selection framework for High-Level Synthesis Design Space Exploration (DSE). It measures how accurately the model recommends the best-performing DSE algorithm for a given benchmark, and assesses the resulting optimization performance (ADRS) and runtime compared to heuristic and reinforcement learning baselines.

Datasets

• MachSuite & Polyhedral Benchmarks — total 29; splits: train (20), test (9)

Metrics

• recommendation_accuracy (primary) — range: percent
  • Percentage of correct algorithm recommendations relative to the total number of test samples.
• ADRS — range: other
  • Algorithm Design Space Recommendation Score; a DSE performance metric where lower values indicate better optimization results (e.g., area, power, latency trade-offs).
• runtime — range: seconds
  • Total execution time in seconds for the DSE process.

Input / output format

Input: Feature representation of a hardware benchmark (e.g., graph structure, design space scale, and behavioral characteristics).

Output: A single recommended DSE algorithm from a predefined candidate set (e.g., NSGA-II, SA, ACO, PSO, Lattice, HGBO-DSE, MOEDA, IRONMAN-PRO, QL-MOEA).

Scoring recipe

def compute_recommendation_accuracy(predictions, gold_labels):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    return correct / len(gold_labels)

# ADRS is computed by the DSE solver; lower is better.
# Returns the average metric across benchmarks.

Common pitfalls

• ADRS is a lower-is-better metric, unlike standard accuracy scores.
• DSE algorithms are stochastic; results may vary across runs, so fixed seeds or multiple runs are necessary for fair comparison.
• Test benchmarks are strictly disjoint from training benchmarks to prevent data leakage and evaluate generalization.

Evidence (verbatim from paper)

To validate the Recommendation accuracy (percentage of correct recommendations relative to the total number of samples) of our proposed algorithm, we conduct comparative evaluations of Sober and against classification methods including Random Forest (RF) (Breiman, 2001), XGBoost (Chen and Guestrin, 2016), K-NN (Cover and Hart, 1967), and SVM (Cortes and Vapnik, 1995). Furthermore, to validate the importance of the PPO component, we conduct experiments using only the Supervised recommendation model (SFM) for algorithm recommendation and recorded the results. As shown in Figure 4, our proposed Sober recommendation model demonstrates significantly more accurate algorithm recommendations compared to traditional classification models, achieving an average improvement of 35.57%.

Citation

@misc{xu2026soberdse,
  title={SoberDSE: Sample-Efficient Design Space Exploration via Learning-Based Algorithm Selection},
  author={Lei Xu et al. (2026)},
  year={2026},
  note={arXiv:2603.00986}
}

• arXiv: 2603.00986