Benchmark Suite Manager

Manage benchmarks for algorithm engineering experiments and evaluations

a5c-ai Updated 1.7k repo stars

File contents

Benchmark Suite Manager

Purpose

Provides expert guidance on managing benchmark suites for algorithm engineering and experimental evaluation.

Capabilities

  • Standard benchmark suite access (DIMACS, TSPLIB, etc.)
  • Instance generation for specific problem classes
  • Statistical analysis of results
  • Performance comparison tables
  • Visualization of scaling behavior
  • Reproducibility support

Usage Guidelines

  1. Suite Selection: Choose appropriate benchmark suite
  2. Instance Selection: Select representative instances
  3. Execution: Run experiments systematically
  4. Analysis: Perform statistical analysis
  5. Reporting: Generate comparison tables and plots

Tools/Libraries

  • DIMACS
  • TSPLIB
  • SuiteSparse Matrix Collection
  • Statistical tools

a5c-ai/babysitter/tree/main/library/specializations/domains/science/computer-science/skills/benchmark-suite-manager commit c9f8858ae9

Frequently asked questions

npx skillmds@latest add a5c-ai/benchmark-suite-manager-2