Volcano Scheduler Configuration
View Volcano scheduler configuration to understand scheduling policies, enabled plugins, and actions. This skill helps diagnose configuration-related scheduling behaviors.
Scope: This skill is for diagnosis only. It retrieves configuration for analysis but does not modify any cluster state.
Usage
bash skills/core/volcano-scheduler-config/scripts/get-scheduler-config.sh [options]
Parameters
| Parameter | Required | Description |
|---|---|---|
--section SECTION |
no | Show specific section: actions, plugins, tiers, all (default: all) |
--format FORMAT |
no | Output format: yaml, json, summary (default: summary) |
--raw |
no | Show raw ConfigMap data without parsing |
Examples
Get summary of scheduler configuration:
bash skills/core/volcano-scheduler-config/scripts/get-scheduler-config.sh
View actions configuration only:
bash skills/core/volcano-scheduler-config/scripts/get-scheduler-config.sh --section actions
View plugins configuration:
bash skills/core/volcano-scheduler-config/scripts/get-scheduler-config.sh --section plugins
Show full YAML configuration:
bash skills/core/volcano-scheduler-config/scripts/get-scheduler-config.sh --format yaml
Get raw ConfigMap:
bash skills/core/volcano-scheduler-config/scripts/get-scheduler-config.sh --raw
Understanding Scheduler Configuration
Configuration Location
The Volcano scheduler configuration is stored in:
- Namespace:
volcano-system - ConfigMap:
volcano-scheduler-configmap - Key:
volcano-scheduler.conf
Configuration Structure
actions: "enqueue, allocate, backfill" # Pipeline order
tiers:
- plugins:
- name: priority
- name: gang
- name: conformance
- plugins:
- name: overcommit
- name: drf
- name: predicates
- name: proportion
- name: nodeorder
- name: binpack
Actions
Actions define the two-phase scheduling pipeline:
- Enqueue phase — PodGroup is admitted to the queue based on queue capacity (does the queue have enough deserved resources?)
- Allocate phase — Individual pods are placed on nodes based on node resources, affinity, taints, etc.
A job can pass enqueue (PodGroup moves to Inqueue) but fail allocation (pods stay Pending) if node-level constraints block placement. This two-phase model explains the common scenario: "queue has capacity but pods won't schedule."
Actions in order:
| Action | Required | Purpose |
|---|---|---|
enqueue |
Yes | Admit pod groups to queue |
allocate |
Yes | Allocate resources to pods |
backfill |
No | Fill idle resources with best-effort pods |
preempt |
No | Evict low-priority pods for high-priority |
reclaim |
No | Reclaim resources from over-allocated queues |
elect |
No | Select target workload (removed in v1.6+) |
Tiers
Tiers divide plugins into priority levels:
- Tier 1: First to evaluate, results cannot be overridden
- Tier 2: Second to evaluate, lower priority
Common tier organization:
- Tier 1: Critical plugins (priority, gang, conformance)
- Tier 2: Resource and optimization plugins
Plugins
Critical Plugins
| Plugin | Function | Critical For |
|---|---|---|
gang |
Gang scheduling | Batch jobs, ML training |
priority |
Priority sorting | Workload prioritization |
conformance |
Protect critical pods | System stability |
Resource Plugins
| Plugin | Function | Use Case |
|---|---|---|
proportion |
Fair share allocation | Multi-tenant clusters |
drf |
Dominant Resource Fairness | Fair GPU/CPU sharing |
overcommit |
Allow overcommit | Resource efficiency |
Node Plugins
| Plugin | Function | Use Case |
|---|---|---|
predicates |
Node filtering | Resource/affinity matching |
nodeorder |
Node ranking | Node selection optimization |
binpack |
Dense packing | Reduce fragmentation |
numaaware |
NUMA topology | HPC workloads |
Queue Plugins
| Plugin | Function | Use Case |
|---|---|---|
priority |
Queue ordering | Queue prioritization |
proportion |
Proportional share | Fair queue allocation |
Diagnostic Use Cases
Case 1: Check If Gang Scheduling is Enabled
bash skills/core/volcano-scheduler-config/scripts/get-scheduler-config.sh --section plugins
Look for name: gang in the plugin list. If missing, Gang scheduling will not work.
Case 2: Verify Reclaim is Enabled
bash skills/core/volcano-scheduler-config/scripts/get-scheduler-config.sh --section actions
Look for reclaim in the actions list. If missing, queues cannot reclaim resources.
Case 3: Check Proportion Plugin Configuration
bash skills/core/volcano-scheduler-config/scripts/get-scheduler-config.sh --section plugins
Look for name: proportion. If missing, queue fair-share allocation is disabled.
Case 4: Compare with Default Configuration
Default configuration:
actions: "enqueue, allocate, backfill"
tiers:
- plugins:
- name: priority
- name: gang
enablePreemptable: false
- name: conformance
- plugins:
- name: overcommit
- name: drf
enablePreemptable: false
- name: predicates
- name: proportion
- name: nodeorder
- name: binpack
If your configuration differs significantly, it may explain scheduling behaviors.
Case 5: Check Plugin Arguments
Some plugins accept arguments:
- name: overcommit
arguments:
overcommit-factor: "1.2"
Use --format yaml or --raw to see full plugin configurations including arguments.
Common Configuration Issues
Issue 1: Missing Gang Plugin
Symptom: PodGroups stay Pending, minMember never satisfied
Check:
bash skills/core/volcano-scheduler-config/scripts/get-scheduler-config.sh --section plugins | grep gang
Solution: Add gang plugin to configuration
Issue 2: Missing Reclaim Action
Symptom: Queues cannot reclaim resources from over-allocated queues
Check:
bash skills/core/volcano-scheduler-config/scripts/get-scheduler-config.sh --section actions | grep reclaim
Solution: Add reclaim to actions list
Issue 3: Wrong Action Order
Symptom: Unexpected scheduling behavior
Check:
bash skills/core/volcano-scheduler-config/scripts/get-scheduler-config.sh --section actions
Common orders:
- Default:
enqueue, allocate, backfill - With preemption:
enqueue, allocate, backfill, preempt - With reclaim:
enqueue, allocate, backfill, reclaim - Full:
enqueue, allocate, backfill, preempt, reclaim
Important: enqueue must come before allocate. allocate must be present.
Issue 4: Plugin Tier Misconfiguration
Symptom: Priority/preemption not working as expected
Check:
bash skills/core/volcano-scheduler-config/scripts/get-scheduler-config.sh --section tiers
Guideline:
- Tier 1: Plugins that should not be overridden (priority, gang)
- Tier 2: Resource optimization plugins (proportion, binpack)
Configuration Options Reference
Plugin Arguments
| Plugin | Argument | Default | Description |
|---|---|---|---|
gang |
enablePreemptable |
true |
Allow Gang pods to be preempted |
overcommit |
overcommit-factor |
1.2 |
Multiplier for allocatable resources |
drf |
enablePreemptable |
true |
Allow DRF pods to be preempted |
nodeorder |
various weights | 0 |
Node scoring weights |
proportion |
(none) | - | No arguments |
predicates |
GPUSharingEnable |
false |
Enable GPU sharing |
Action-Specific Notes
Enqueue
- Evaluates
jobEnqueueableFnfrom plugins - Default
overcommit-factorof 1.2 means 20% overcommit
Allocate
- Main allocation logic
- Calls
allocateaction for each tier
Backfill
- Only schedules best-effort pods (no resource requests)
- Ignores queue deserved resources
Preempt
- Requires
priorityplugin for comparison - Requires
preemptableFnfrom plugins
Reclaim
- Requires
proportionplugin - Requires
reclaimableFnfrom plugins - Reclaims from over-allocated queues
Environment Variables
| Variable | Default | Description |
|---|---|---|
VOLCANO_SCHEDULER_NS |
volcano-system |
Scheduler namespace |
VOLCANO_SCHEDULER_CONFIG |
volcano-scheduler-configmap |
ConfigMap name |
Output Formats
Summary Format (default)
Human-readable summary:
Scheduler Configuration
=======================
Actions: enqueue, allocate, backfill
Tier 1 Plugins:
- priority
- gang
- conformance
Tier 2 Plugins:
- overcommit
- drf
- predicates
- proportion
- nodeorder
- binpack
YAML Format
Parsed YAML structure:
actions: "enqueue, allocate, backfill"
tiers:
- plugins:
- name: priority
- name: gang
JSON Format
Machine-parseable:
{
"actions": "enqueue, allocate, backfill",
"tiers": [
{
"plugins": [
{"name": "priority"},
{"name": "gang"}
]
}
]
}
Raw Format
ConfigMap raw output:
volcano-scheduler.conf:
---
actions: "enqueue, allocate, backfill"
tiers:
...
See Also
volcano-queue-diagnose- Queue resource analysisvolcano-gang-scheduling- Gang scheduling issuesvolcano-diagnose-pod- Pod scheduling diagnosisvolcano-scheduler-logs- Scheduler decision logs