# Hpc Runtime Doctor

> Diagnose HPC runtime and scheduler problems for materials simulations, including MPI/OpenMP/GPU layout, modules, CUDA/Kokkos hints, scratch paths, walltime, job arrays, restart strategy, scheduler portability, and resource mismatch. Use when jobs fail, run slowly, get killed, or behave differently on a cluster than on a workstation.

- Skill: `zuoguyoupan2023/hpc-runtime-doctor` (Agent Skill, multi-file: 3 files)
- Install (CLI): `npx skillmds@latest add zuoguyoupan2023/hpc-runtime-doctor`
- Raw SKILL.md: https://api.skillmd.com/api/skills/zuoguyoupan2023/hpc-runtime-doctor/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: zuoguyoupan2023 (https://skillmd.com/u/zuoguyoupan2023)
- Updated: 2026-09-22
- Page: https://skillmd.com/skills/zuoguyoupan2023/hpc-runtime-doctor

---


# HPC Runtime Doctor

## Goal

Turn cluster symptoms into a resource-layout diagnosis, environment checklist, and safe retry plan.

## Requirements

- Python 3.10+
- No external dependencies
- Works on Linux, macOS, and Windows

## Inputs to Gather

| Input | Description | Example |
|-------|-------------|---------|
| Scheduler | SLURM, PBS, LSF, local | `slurm` |
| Nodes/tasks/threads | Runtime layout | `2 nodes, 128 tasks, 2 threads` |
| GPUs | GPUs requested | `4` |
| Symptoms | Observed failure | `oom,killed,slow-gpu` |
| MPI/OpenMP/GPU use | Parallel modes | `mpi+openmp+gpu` |
| Walltime | Requested time | `12:00:00` |
| Scratch | Whether scratch is used | `true` |

## Decision Guidance

- Check resource layout before changing physics settings.
- Confirm module/compiler/MPI/CUDA consistency before debugging solver behavior.
- Treat missing restart files and scratch cleanup as workflow failures, not physics failures.
- For GPU jobs, confirm the executable was built with the requested accelerator backend.

## Script Outputs

`scripts/hpc_runtime_doctor.py` emits:

- `resource_layout`
- `diagnoses`
- `environment_checks`
- `retry_plan`
- `scheduler_notes`

## Workflow

```bash
python3 skills/hpc-deployment/hpc-runtime-doctor/scripts/hpc_runtime_doctor.py \
  --scheduler slurm \
  --nodes 2 \
  --tasks 128 \
  --cpus-per-task 2 \
  --gpus 4 \
  --symptoms oom,slow-gpu \
  --uses-mpi \
  --uses-openmp \
  --uses-gpu \
  --json
```

## Error Handling

Invalid resource counts stop with exit code 2. Unknown symptoms are preserved as custom items for human review.

## Limitations

This skill does not query a live scheduler. It diagnoses from the submitted layout and symptoms.

## Security

- Inputs are scalar CLI values and booleans only.
- The script does not execute scheduler commands or inspect environment variables.
- The skill uses `Bash` only to run its bundled script.

## References

- See `references/hpc_runtime_patterns.md` for scheduler and runtime diagnosis patterns.

## Version History

- 1.0.0: Initial HPC runtime diagnosis skill.