Compileiq Run Search

compileiq-run-search

After you have an objective function (from compileiq-author-objective) and a search space (from compileiq-search-space), this skill helps you choose the worker, size the configuration, and run the search safely.

When

• About to instantiate Search(...) and call .start().
• Search is converging too fast or too slow and the user is unsure how to re-size pool/generations.
• Search hangs on individual configs and the worker doesn't kill them.
• Scaling out from one GPU to a Ray cluster.

Worker selection

Pass either a built-in WorkerTypes enum value or the worker class itself to Search(worker_type=...):

from compileiq.types import WorkerTypes
from compileiq.worker import (
    MultiProcessWorker,    # default
    IsoMultiProcessWorker, # spawns fresh process per task; kill-safe
    RayWorker,             # distributed
    AsyncWorker,           # asyncio for async def objectives
)

Default recommendation for compiler tuning of GPU kernels: IsoMultiProcessWorker with task_timeout between 30s (small kernels) and 180s (large attention / XLA HLO).

SearchConfiguration sizing

Reference: compileiq/types.py:473-615. Defaults auto-derive; only set what you must.

from compileiq.types import SearchConfiguration, ProblemType

config = SearchConfiguration(
    problem_type=ProblemType.MIN,   # MIN for latency; MAX for throughput
    generations=10,                  # required, > 0
    pool_size=15,                    # > 5; auto-derives if omitted
    # cull_size auto-derives to 75% of pool, rounded down to even
    # mutate_rate defaults to 0.25
    # num_objectives defaults to 1
    # normalize defaults to False (set True for cross-GPU runs)
)

Sanity rule of thumb: if pool_size * generations < 50, you are exploring, not optimizing. If > 2000, you are probably overfitting to measurement noise — compileiq-validate-result will earn its keep there.

Search(...) constructor — every relevant kwarg

from pathlib import Path
from compileiq.ciq import Search
from compileiq.search_spaces.compilers import PtxasSearchSpace
from compileiq.tracker import LoguruTrackerConfig

tuner = Search(
    objective_function=objective,
    search_space=PtxasSearchSpace(version="13.3", variant="att"),
    search_config=config,
    worker_type=IsoMultiProcessWorker,                 # or WorkerTypes.ISOLATED
    tracker_config=LoguruTrackerConfig(sink="optimization.log"),
    dump_results=Path("results.csv"),                  # ALWAYS set this
    cache_folder=None,                                  # default ~/.cache/compileiq
    disable_progress_bar=False,
    exit_on_failure=True,
    debug=False,
)

Always set dump_results=Path(...). CSV is flushed every batch, so a crashed or killed run leaves recoverable state.

start(...) semantics

results = tuner.start(num_workers=4, task_timeout=120)

• num_workers: ignored by workers where respects_num_workers=False (RayWorker, AsyncWorker); CompileIQ emits the warning "num_workers is not supported by <WorkerName>" (compileiq/ciq.py:449-451) so users recognize it.
• task_timeout: ignored where supports_timeout=False (RayWorker). Critical for IsoMultiProcessWorker — without it a hung config wedges that branch.
• Returns a SearchResult. Don't process inline; hand off to compileiq-validate-result.

Tracker choice (one-line each)

from compileiq.tracker import DisabledTrackerConfig, LoguruTrackerConfig, MLflowTrackerConfig

• DisabledTrackerConfig() — default, no overhead. Fine for one-off runs.
• LoguruTrackerConfig(sink="optimization.log", level="INFO") — recommended for serious campaigns. Negligible overhead.
• MLflowTrackerConfig(experiment_name="...", tracking_uri="...", run_name="...") — when integrating with ML Ops; creates a nested MLflow run per evaluation.

Sample before you search

Search.sample(n) returns n randomly sampled parameter dicts from the search space without running the search. Use it to:

1. Confirm the search space resolves at all (cheaper than the bootstrap round-trip; uses the in-memory state of Search).
2. Eyeball that the dicts have the keys your objective expects.
3. Feed a single sample into the objective by hand to verify it runs.

sample = tuner.sample(1)[0]
print(sample)
print(objective(sample))   # should return a real float, not raise

GPU clock locking (operator-level)

Stable measurements need locked clocks. Lock before tuner.start(), unlock via atexit. Requires sudo.

sudo nvidia-smi -pm 1
MAX_GPU=$(nvidia-smi --query-gpu=clocks.max.graphics --format=csv,noheader,nounits | head -1)
MAX_MEM=$(nvidia-smi --query-gpu=clocks.max.memory --format=csv,noheader,nounits | head -1)
sudo nvidia-smi --lock-gpu-clocks=$MAX_GPU,$MAX_GPU --lock-memory-clocks=$MAX_MEM,$MAX_MEM

import atexit, subprocess
def unlock():
    subprocess.run(["sudo", "nvidia-smi", "--reset-gpu-clocks", "--reset-memory-clocks"],
                   check=False)
atexit.register(unlock)

Inside a CI container or a shared cluster where sudo isn't available, skip this; report higher CV% to the validation skill so it knows to compensate.

Self-test

python scripts/smoke_search.py

Runs a 2-generation search on x**2 + y with MultiProcessWorker and verifies results.get_best_result() returns a dict with score_1 and params.

Gotchas

• Forgetting task_timeout with IsoMultiProcessWorker is the most common reason a search hangs for hours. The worker will kill a stuck process but only after task_timeout elapses.
• forkserver issues on some hosts manifest as EOFError or "Broken pipe" on the first eval. Set CIQ_PROCESS_MODE=spawn.
• num_workers > num_gpus is fine for fast CPU-side objectives but oversubscribes GPUs for kernel objectives. For GPU kernels: pin CUDA_VISIBLE_DEVICES inside the objective and set num_workers = num_gpus.
• Don't put GPU-clock lock calls inside the objective. They require sudo and are per-host operator setup, not per-eval.

Next

• After .start() returns: compileiq-validate-result.
• If something's wrong: compileiq-debug.