Provision with MAAS
Keep MAAS authoritative for machine lifecycle, power, PXE/DHCP, OS images,
commissioning, deployment, release, pools, zones, and tags. Use DeepOps to
install a controller, request explicit deploy/tag/release operations, and turn
deployed machine tags into Ansible inventory.
Preconditions
- Work from the DeepOps repository root on a prepared Ansible control machine.
- Confirm that every target is authorized for reinstallation. A deploy erases
its installed OS, and
maas_deploy.sh releases every listed deployed machine
before installing it again.
- Provide a MAAS 3.x controller reachable through its REST API and an OAuth1
API key in
consumer_key:token_key:token_secret format.
- Ensure the provisioning VLAN can safely use MAAS DHCP, with a reserved
dynamic range that does not overlap site addresses. Targets must PXE boot on
that network.
- Configure target power control in MAAS, or arrange manual power cycles.
Commission each target to
Ready, sync the requested Ubuntu image, and add
an SSH public key that the operator can use after deployment.
- Use Ubuntu 24.04 with the MAAS 3.7 repository for a new controller, or the
MAAS 3.5 repository while the controller remains on Ubuntu 22.04.
Procedure
Prepare MAAS and DeepOps
Install DeepOps dependencies and create the private configuration if this
checkout has not been prepared:
./scripts/setup.sh
If DeepOps must install the MAAS controller, set maas_adminusers,
maas_dns_domain, maas_region_controller,
maas_region_controller_url, and maas_repo in
config/group_vars/all.yml. Review the inventory, then limit the
controller playbook to exactly that host:
ansible-playbook -l <maas-controller-host> playbooks/provisioning/maas.yml
Skip this step when using an existing MAAS controller.
Edit config/maas-inventory.yml, which scripts/setup.sh creates from
config.example/maas-inventory.yml:
api_url: "http://maas-server:5240/MAAS/api/2.0"
api_key: "consumer_key:token_key:token_secret"
ssh_user: "ubuntu"
network: "192.0.2"
ssh_bastion: "user@bastion-host"
machines: "node01 node02 node03"
Omit network for first-address selection and ssh_bastion for direct
access. Set machines to the exact MAAS hostnames this operation owns; do
not rely on the script's test-VM defaults. Environment variables override
the file. MAAS_INVENTORY_CONFIG selects another inventory config for the
Python inventory script; the deploy script always reads the repo-local file
but accepts MAAS_API_URL, MAAS_API_KEY, MAAS_MACHINES,
MAAS_SSH_USER, MAAS_NETWORK, and SSH proxy overrides.
Review the resolved targets and their current state before any mutation:
MAAS_MACHINES="node01 node02 node03" ./scripts/maas_deploy.sh --status
Require every intended hostname to appear. Check status, IP, OS, and tags;
stop if an unexpected machine is present.
Map MAAS tags to inventory
scripts/maas_inventory.py includes only machines whose MAAS status code is
6 (Deployed). It uses each hostname as the Ansible host and maps every
MAAS tag to an Ansible group. Tag leaf groups; the script creates the DeepOps
parent/child relationships.
Use these canonical role tags:
| Cluster |
MAAS leaf tags |
| Kubernetes |
kube_control_plane, kube_node, etcd |
| Slurm |
slurm-master, slurm-node, optionally slurm-nfs, slurm-cache, slurm-metric, slurm-login |
Legacy kube-master, kube-node, and k8s-cluster tags are accepted and
aliased to canonical group names. The deploy helper does not remove these
legacy tags when applying a profile, so remove them manually in MAAS before a
role change. Otherwise a target can remain in Kubernetes inventory groups after
switching to the Slurm profile. Other tags become same-named Ansible groups.
For the built-in profiles, ordering is significant: the first hostname in
machines becomes the Kubernetes control-plane/etcd node or Slurm master;
the remaining hosts become Kubernetes or Slurm workers. Apply or change a
profile without reinstalling:
MAAS_MACHINES="node01 node02 node03" \
./scripts/maas_deploy.sh --profile k8s --tags-only
Before applying the profile, this command removes the canonical Kubernetes and
Slurm test tags from every listed target. It does not remove legacy Kubernetes
tags or unrelated MAAS tags. Use MAAS directly for a custom role layout, for
example:
maas admin tag update-nodes slurm-master add=<node01-system-id>
maas admin tag update-nodes slurm-node add=<node02-system-id> add=<node03-system-id>
Deploy
Run only after reviewing the exact targets and accepting a reinstall of all of
them:
MAAS_MACHINES="node01 node02 node03" \
./scripts/maas_deploy.sh --os noble --profile slurm
Use --os jammy for Ubuntu 22.04. The default is noble. The script:
- Resolves every hostname to a MAAS system ID and stops if any is absent.
- Releases deployed machines and waits up to 600 seconds for every machine to
reach
Ready.
- Requests deployment of the selected Ubuntu series and waits up to 600
seconds for every machine to reach
Deployed.
- Selects an IP matching
network, falling back to the first MAAS address,
and waits up to 120 seconds for SSH as ssh_user.
- Reports
lsb_release -ds, then clears known test tags and applies the
selected profile.
For manual power control, perform each power-on requested by MAAS while the
script is waiting.
Validate after provisioning
Confirm MAAS state, selected IPs, OS, and tags:
MAAS_MACHINES="node01 node02 node03" ./scripts/maas_deploy.sh --status
Require every target to show Deployed, the requested Ubuntu release, a
usable target-network IP, and the intended role tags.
Inspect the dynamic inventory before running a cluster playbook:
./scripts/maas_inventory.py --list
ansible-inventory -i scripts/maas_inventory.py --graph
This inventory is controller-wide: it includes every Deployed machine
visible through the configured MAAS API, not only the hostnames in
machines or MAAS_MACHINES. Require each operation-owned target to be
present with the expected leaf groups and correct first-host/worker split;
unrelated deployed machines may also appear in all. Inspect
ansible_host and maas_* host variables in --list when anything is
unexpected.
Prove Ansible can reach the operation-owned machines only:
ansible -i scripts/maas_inventory.py 'node01:node02:node03' -m ping
Build the host pattern from the same reviewed hostname list used for the
deploy or tag operation; do not target all on a shared controller. Require
SUCCESS and "ping": "pong" for every operation-owned target. This
validates provisioning and control-plane access; it does not validate a
Slurm or Kubernetes deployment.
Reruns and role changes
- Rerun
--status, inventory inspection, and Ansible ping freely; they do not
request lifecycle changes.
- Use
--tags-only to change between the built-in Kubernetes and Slurm layouts
without reinstalling. Recheck hostname ordering first because it selects the
control-plane or master node. Inspect each target for legacy kube-master,
kube-node, or k8s-cluster tags and remove them manually in MAAS before the
change; --tags-only does not clear them.
- Treat a rerun of the deploy command as a fresh reinstall, not an idempotent
resume. It releases all listed deployed machines, including machines that
succeeded during a partially failed multi-machine run.
- A released machine that is still commissioned and
Ready can be deployed
again without commissioning it again.
Cleanup
Release only the exact machines owned by the operation:
MAAS_MACHINES="node01 node02 node03" ./scripts/maas_deploy.sh --release
Require All machines released. and verify --status shows Ready. Released
machines disappear from dynamic inventory because it includes only Deployed
machines. Release does not clear their MAAS tags; remove residual role tags in
MAAS separately if the machines are being reassigned, after confirming tag
ownership.
Failure branches
- Inventory returns only an empty
_meta object without an error: the
configuration is missing or still contains example placeholders. Populate
api_url and api_key, or set the matching environment variables, then
rerun --list.
ERROR: MAAS_API_URL not configured or ERROR: MAAS_API_KEY not configured: populate the missing deploy configuration or its environment
override. The deploy helper does not use the inventory helper's graceful
empty-inventory behavior.
MAAS API error: 401: replace or correct MAAS_API_KEY. If the deploy
helper reports that the key must use
consumer_key:token_key:token_secret, correct its three-part format before
making another request.
MAAS connection error: verify MAAS_API_URL and reachability from the
control machine. Do not continue to lifecycle operations until inventory or
status reads succeed.
Machine '<name>' not found in MAAS: compare machines or
MAAS_MACHINES with MAAS hostnames. Correct the target list; do not replace
it with a broader list.
- Inventory is valid but a target is absent: only
Deployed machines with
a non-empty hostname are included. Check --status; finish deployment, or
expect a released/Ready machine to remain absent.
- Inventory chooses the wrong address or SSH times out after 120 seconds:
set
network/MAAS_NETWORK to the target subnet, verify ssh_user, the
imported MAAS SSH key, and ssh_bastion or proxy settings, then rerun the
read-only ping checks. A machine with no MAAS address cannot yield a usable
ansible_host.
- A machine does not reach
Ready or Deployed within 600 seconds: inspect
its MAAS state and events. For manual power, perform the requested power
cycle. If PXE never begins, verify PXE boot and that DHCP is enabled on the
correct VLAN with a non-overlapping dynamic range. Rerun deployment only
after deciding whether reinstalling every listed target is acceptable.
--tags-only fails immediately: supply --profile k8s or
--profile slurm; other profile names are rejected. If groups are still
wrong, inspect tag names and target ordering. Remove legacy kube-master,
kube-node, and k8s-cluster tags manually in MAAS because reapplying a
profile will not clear them; then reapply the intended profile and regenerate
inventory.
1---2name: provision-with-maas3description: Provision or reinstall bare-metal servers and test VMs through Canonical MAAS, map deployed machines into DeepOps Ansible inventory with MAAS tags, validate access, or release them safely. Use when operating DeepOps with a MAAS-owned machine lifecycle.4metadata:5 source: https://skillmd.com/skills/nvidia/provision-with-maas6---78# Provision with MAAS910Keep MAAS authoritative for machine lifecycle, power, PXE/DHCP, OS images,11commissioning, deployment, release, pools, zones, and tags. Use DeepOps to12install a controller, request explicit deploy/tag/release operations, and turn13deployed machine tags into Ansible inventory.1415## Preconditions1617- Work from the DeepOps repository root on a prepared Ansible control machine.18- Confirm that every target is authorized for reinstallation. A deploy erases19 its installed OS, and `maas_deploy.sh` releases every listed deployed machine20 before installing it again.21- Provide a MAAS 3.x controller reachable through its REST API and an OAuth122 API key in `consumer_key:token_key:token_secret` format.23- Ensure the provisioning VLAN can safely use MAAS DHCP, with a reserved24 dynamic range that does not overlap site addresses. Targets must PXE boot on25 that network.26- Configure target power control in MAAS, or arrange manual power cycles.27 Commission each target to `Ready`, sync the requested Ubuntu image, and add28 an SSH public key that the operator can use after deployment.29- Use Ubuntu 24.04 with the MAAS 3.7 repository for a new controller, or the30 MAAS 3.5 repository while the controller remains on Ubuntu 22.04.3132## Procedure3334### Prepare MAAS and DeepOps35361. Install DeepOps dependencies and create the private configuration if this37 checkout has not been prepared:3839 ```bash40 ./scripts/setup.sh41 ```42432. If DeepOps must install the MAAS controller, set `maas_adminusers`,44 `maas_dns_domain`, `maas_region_controller`,45 `maas_region_controller_url`, and `maas_repo` in46 `config/group_vars/all.yml`. Review the inventory, then limit the47 controller playbook to exactly that host:4849 ```bash50 ansible-playbook -l <maas-controller-host> playbooks/provisioning/maas.yml51 ```5253 Skip this step when using an existing MAAS controller.54553. Edit `config/maas-inventory.yml`, which `scripts/setup.sh` creates from56 `config.example/maas-inventory.yml`:5758 ```yaml59 api_url: "http://maas-server:5240/MAAS/api/2.0"60 api_key: "consumer_key:token_key:token_secret"61 ssh_user: "ubuntu"62 network: "192.0.2"63 ssh_bastion: "user@bastion-host"64 machines: "node01 node02 node03"65 ```6667 Omit `network` for first-address selection and `ssh_bastion` for direct68 access. Set `machines` to the exact MAAS hostnames this operation owns; do69 not rely on the script's test-VM defaults. Environment variables override70 the file. `MAAS_INVENTORY_CONFIG` selects another inventory config for the71 Python inventory script; the deploy script always reads the repo-local file72 but accepts `MAAS_API_URL`, `MAAS_API_KEY`, `MAAS_MACHINES`,73 `MAAS_SSH_USER`, `MAAS_NETWORK`, and SSH proxy overrides.74754. Review the resolved targets and their current state before any mutation:7677 ```bash78 MAAS_MACHINES="node01 node02 node03" ./scripts/maas_deploy.sh --status79 ```8081 Require every intended hostname to appear. Check status, IP, OS, and tags;82 stop if an unexpected machine is present.8384### Map MAAS tags to inventory8586`scripts/maas_inventory.py` includes only machines whose MAAS status code is87`6` (`Deployed`). It uses each hostname as the Ansible host and maps every88MAAS tag to an Ansible group. Tag leaf groups; the script creates the DeepOps89parent/child relationships.9091Use these canonical role tags:9293| Cluster | MAAS leaf tags |94|---------|----------------|95| Kubernetes | `kube_control_plane`, `kube_node`, `etcd` |96| Slurm | `slurm-master`, `slurm-node`, optionally `slurm-nfs`, `slurm-cache`, `slurm-metric`, `slurm-login` |9798Legacy `kube-master`, `kube-node`, and `k8s-cluster` tags are accepted and99aliased to canonical group names. The deploy helper does not remove these100legacy tags when applying a profile, so remove them manually in MAAS before a101role change. Otherwise a target can remain in Kubernetes inventory groups after102switching to the Slurm profile. Other tags become same-named Ansible groups.103104For the built-in profiles, ordering is significant: the first hostname in105`machines` becomes the Kubernetes control-plane/etcd node or Slurm master;106the remaining hosts become Kubernetes or Slurm workers. Apply or change a107profile without reinstalling:108109```bash110MAAS_MACHINES="node01 node02 node03" \111 ./scripts/maas_deploy.sh --profile k8s --tags-only112```113114Before applying the profile, this command removes the canonical Kubernetes and115Slurm test tags from every listed target. It does not remove legacy Kubernetes116tags or unrelated MAAS tags. Use MAAS directly for a custom role layout, for117example:118119```bash120maas admin tag update-nodes slurm-master add=<node01-system-id>121maas admin tag update-nodes slurm-node add=<node02-system-id> add=<node03-system-id>122```123124### Deploy125126Run only after reviewing the exact targets and accepting a reinstall of all of127them:128129```bash130MAAS_MACHINES="node01 node02 node03" \131 ./scripts/maas_deploy.sh --os noble --profile slurm132```133134Use `--os jammy` for Ubuntu 22.04. The default is `noble`. The script:1351361. Resolves every hostname to a MAAS system ID and stops if any is absent.1372. Releases deployed machines and waits up to 600 seconds for every machine to138 reach `Ready`.1393. Requests deployment of the selected Ubuntu series and waits up to 600140 seconds for every machine to reach `Deployed`.1414. Selects an IP matching `network`, falling back to the first MAAS address,142 and waits up to 120 seconds for SSH as `ssh_user`.1435. Reports `lsb_release -ds`, then clears known test tags and applies the144 selected profile.145146For manual power control, perform each power-on requested by MAAS while the147script is waiting.148149### Validate after provisioning1501511. Confirm MAAS state, selected IPs, OS, and tags:152153 ```bash154 MAAS_MACHINES="node01 node02 node03" ./scripts/maas_deploy.sh --status155 ```156157 Require every target to show `Deployed`, the requested Ubuntu release, a158 usable target-network IP, and the intended role tags.1591602. Inspect the dynamic inventory before running a cluster playbook:161162 ```bash163 ./scripts/maas_inventory.py --list164 ansible-inventory -i scripts/maas_inventory.py --graph165 ```166167 This inventory is controller-wide: it includes every `Deployed` machine168 visible through the configured MAAS API, not only the hostnames in169 `machines` or `MAAS_MACHINES`. Require each operation-owned target to be170 present with the expected leaf groups and correct first-host/worker split;171 unrelated deployed machines may also appear in `all`. Inspect172 `ansible_host` and `maas_*` host variables in `--list` when anything is173 unexpected.1741753. Prove Ansible can reach the operation-owned machines only:176177 ```bash178 ansible -i scripts/maas_inventory.py 'node01:node02:node03' -m ping179 ```180181 Build the host pattern from the same reviewed hostname list used for the182 deploy or tag operation; do not target `all` on a shared controller. Require183 `SUCCESS` and `"ping": "pong"` for every operation-owned target. This184 validates provisioning and control-plane access; it does not validate a185 Slurm or Kubernetes deployment.186187### Reruns and role changes188189- Rerun `--status`, inventory inspection, and Ansible ping freely; they do not190 request lifecycle changes.191- Use `--tags-only` to change between the built-in Kubernetes and Slurm layouts192 without reinstalling. Recheck hostname ordering first because it selects the193 control-plane or master node. Inspect each target for legacy `kube-master`,194 `kube-node`, or `k8s-cluster` tags and remove them manually in MAAS before the195 change; `--tags-only` does not clear them.196- Treat a rerun of the deploy command as a fresh reinstall, not an idempotent197 resume. It releases all listed deployed machines, including machines that198 succeeded during a partially failed multi-machine run.199- A released machine that is still commissioned and `Ready` can be deployed200 again without commissioning it again.201202### Cleanup203204Release only the exact machines owned by the operation:205206```bash207MAAS_MACHINES="node01 node02 node03" ./scripts/maas_deploy.sh --release208```209210Require `All machines released.` and verify `--status` shows `Ready`. Released211machines disappear from dynamic inventory because it includes only `Deployed`212machines. Release does not clear their MAAS tags; remove residual role tags in213MAAS separately if the machines are being reassigned, after confirming tag214ownership.215216## Failure branches217218- **Inventory returns only an empty `_meta` object without an error:** the219 configuration is missing or still contains example placeholders. Populate220 `api_url` and `api_key`, or set the matching environment variables, then221 rerun `--list`.222- **`ERROR: MAAS_API_URL not configured` or `ERROR: MAAS_API_KEY not223 configured`:** populate the missing deploy configuration or its environment224 override. The deploy helper does not use the inventory helper's graceful225 empty-inventory behavior.226- **`MAAS API error: 401`:** replace or correct `MAAS_API_KEY`. If the deploy227 helper reports that the key must use228 `consumer_key:token_key:token_secret`, correct its three-part format before229 making another request.230- **`MAAS connection error`:** verify `MAAS_API_URL` and reachability from the231 control machine. Do not continue to lifecycle operations until inventory or232 status reads succeed.233- **`Machine '<name>' not found in MAAS`:** compare `machines` or234 `MAAS_MACHINES` with MAAS hostnames. Correct the target list; do not replace235 it with a broader list.236- **Inventory is valid but a target is absent:** only `Deployed` machines with237 a non-empty hostname are included. Check `--status`; finish deployment, or238 expect a released/`Ready` machine to remain absent.239- **Inventory chooses the wrong address or SSH times out after 120 seconds:**240 set `network`/`MAAS_NETWORK` to the target subnet, verify `ssh_user`, the241 imported MAAS SSH key, and `ssh_bastion` or proxy settings, then rerun the242 read-only ping checks. A machine with no MAAS address cannot yield a usable243 `ansible_host`.244- **A machine does not reach `Ready` or `Deployed` within 600 seconds:** inspect245 its MAAS state and events. For manual power, perform the requested power246 cycle. If PXE never begins, verify PXE boot and that DHCP is enabled on the247 correct VLAN with a non-overlapping dynamic range. Rerun deployment only248 after deciding whether reinstalling every listed target is acceptable.249- **`--tags-only` fails immediately:** supply `--profile k8s` or250 `--profile slurm`; other profile names are rejected. If groups are still251 wrong, inspect tag names and target ordering. Remove legacy `kube-master`,252 `kube-node`, and `k8s-cluster` tags manually in MAAS because reapplying a253 profile will not clear them; then reapply the intended profile and regenerate254 inventory.