Ansible

Ansible Automation

Project Layout

Roles are the primary organizational unit. Use dimensional groups (what/where/when) in a single inventory and target with --limit.

ansible-project/
  ansible.cfg
  requirements.yml          # pinned collection/role versions
  site.yml                  # master playbook (imports tier playbooks)
  playbooks/
    webservers.yml
    dbservers.yml
  inventory/
    hosts.yml               # dimensional groups: what/where/when
    group_vars/
      all/
        vars.yml            # plaintext variables
        vault.yml           # ansible-vault encrypted
      webservers.yml
    host_vars/
      db01.yml
  roles/
    common/
      defaults/main.yml     # user-overridable defaults (low precedence)
      tasks/main.yml
      handlers/main.yml
      templates/
      files/
      vars/main.yml         # internal constants (high precedence)
      meta/main.yml
    nginx/
    myapp/
  molecule/                 # or per-role molecule/ dirs
  .ansible-lint
  .yamllint

• group_vars/{group}/vars.yml + vault.yml keeps variable names visible while values stay encrypted (vault indirection pattern).
• site.yml imports tier playbooks; run ansible-playbook site.yml --limit webservers for targeted execution.

Inventory Management

Prefer YAML over INI -- INI has inconsistent type parsing between inline host vars and :vars sections.

Always use aliases for hosts. The left-hand inventory name is what you reference in playbooks, logs, and --limit; ansible_host holds the actual connection target. This decouples automation logic from infrastructure details.

Organize hosts along three dimensions -- function (webservers, dbservers), location (east, west), and lifecycle (prod, staging) -- then target with intersection patterns (--limit "webservers:&prod") instead of creating combinatorial group names. Use children: only when one group genuinely contains another.

Debug inventory with ansible-inventory --graph (hierarchy), --list (full JSON), or --host <name> (merged variables). Add -vvv to trace variable sources.

See references/inventory.md for YAML/INI syntax, aliases, group hierarchies, host patterns, debugging, group design patterns, and common mistakes.

Linting Setup

.ansible-lint

---
profile: moderate  # good starting point; bump to safety for stricter enforcement

exclude_paths:
  - .cache/
  - .git/
  - molecule/

warn_list:
  - experimental

The safety profile is the strictest fully-implemented profile. shared and production contain rules not yet enforced.

Run: ansible-lint (auto-discovers playbooks). Fix auto-fixable violations: ansible-lint --fix.

.yamllint

ansible-lint runs yamllint internally. A separate .yamllint is only needed to customize YAML rules beyond ansible-lint defaults:

---
extends: default
rules:
  line-length:
    max: 160
    level: warning
  truthy:
    allowed-values: ["true", "false"]
    check-keys: false
  indentation:
    spaces: 2
    indent-sequences: true
  comments:
    min-spaces-from-content: 1
  comments-indentation: disable
  document-start: disable
  octal-values:
    forbid-implicit-octal: true
    forbid-explicit-octal: true
  braces:
    min-spaces-inside: 0
    max-spaces-inside: 1

When providing a custom .yamllint, the braces, comments.min-spaces-from-content, and octal-values settings above are required for ansible-lint compatibility.

See references/linting.md for the full rule catalog, profiles, CI/CD integration, and skip mechanisms.

Playbook Authoring

Essentials

Always use FQCN (Fully Qualified Collection Name), name every task, use true/false (not yes/no), quote file modes, set explicit state:

---
- name: Deploy web application
  hosts: webservers
  gather_facts: true

  tasks:
    - name: Install nginx
      ansible.builtin.apt:
        name: nginx
        state: present    # always explicit, even when it's the default
        update_cache: true
        cache_valid_time: 3600
      become: true

    - name: Deploy nginx config
      ansible.builtin.template:
        src: nginx.conf.j2
        dest: /etc/nginx/nginx.conf
        mode: "0644"       # always quoted string
        owner: root
        group: root
      become: true
      notify: Restart nginx

  handlers:
    - name: Restart nginx
      ansible.builtin.systemd:
        name: nginx
        state: restarted
      become: true

Variable precedence

Know these tiers (lowest to highest):

Gotchas:

• defaults/main.yml (tier 1) vs vars/main.yml (tier 5) -- vastly different precedence despite similar locations. Use defaults/ for user-configurable values; vars/ for constants.
• set_fact beats almost everything except extra vars. A stray set_fact can silently override inventory variables.
• Child groups beat parent groups. group_vars/webservers.yml beats group_vars/all.yml.

import_tasks vs include_tasks

Use import_tasks by default. Switch to include_tasks only when you need loops or conditional file selection.

Handlers

• Handlers run once at end of play, even if notified multiple times. Use meta: flush_handlers when a later task depends on the handler having run.
• Handlers only fire when explicitly notified. Listing a second handler below a first in handlers/main.yml does not chain them. A "reporter" handler that's supposed to surface failures from an earlier handler has to be notified by that earlier handler (or collapsed into it).
• Never use variables in handler names -- they template too early and fail. Put variables in handler task parameters instead.
• Use listen to decouple notification name from handler name:

- name: Restart nginx
  ansible.builtin.systemd:
    name: nginx
    state: restarted
  listen: restart web stack

- name: Clear nginx cache
  ansible.builtin.file:
    path: /var/cache/nginx
    state: absent
  listen: restart web stack

One convergence path per resource

For any resource a role manages -- a compose project, a systemd unit, a managed config file -- pick exactly one convergence path and stick to it:

• Option A (in-flow converge): an unconditional task at the end of the role brings the resource to the desired state, relying on the module's own idempotency for no-op runs.
• Option B (handler-only converge): templates/config tasks notify: a handler, and the handler is the only thing that brings the resource up.

Running both paths for the same resource is the recurring bug. Best case, you do redundant work on every change. Worst case -- with partially-idempotent modules like community.docker.docker_compose_v2 -- every apply reports changed with no real drift. Never cross-wire a handler when: to a task-scoped register, and never gate a fallback converge task on not (x is changed); both patterns leak.

Choose option B when the compose file has restart: "no" init containers, when services use depends_on: condition: service_healthy (which state: restarted does not re-evaluate), or when the module documents any partial-idempotency caveat. Use a stop+start pair under a shared listen: so depends_on and healthchecks are re-evaluated. Otherwise option A is simpler.

See references/convergence.md for the full BAD/GOOD examples, the docker_compose_v2 init-container analysis, and the stop+start handler pattern.

Error handling

- name: Risky deployment with rollback
  block:
    - name: Deploy new version
      ansible.builtin.copy:
        src: app-v2.tar.gz
        dest: /opt/app/app.tar.gz
        mode: "0644"

    - name: Extract and restart
      ansible.builtin.shell: |
        cd /opt/app && tar xzf app.tar.gz && ./restart.sh
      changed_when: true

  rescue:
    - name: Rollback to previous version
      ansible.builtin.copy:
        src: /opt/app/backup/app.tar.gz
        dest: /opt/app/app.tar.gz
        mode: "0644"

  always:
    - name: Ensure service is running
      ansible.builtin.systemd:
        name: myapp
        state: started

Avoid ignore_errors: true -- it masks real failures. Use failed_when: false when you truly want to suppress all errors, as it is explicit about intent.

Role Composition

Roles compose through three mechanisms: meta/main.yml dependencies, the play-level roles: keyword, and include_role/import_role in tasks.

Reserve meta/main.yml dependencies for hard, unconditional prerequisites -- role B genuinely cannot function without role A. For everything else, sequence roles at the playbook level: meta deps cannot be gated with when:, and role deduplication happens before variable evaluation, so two invocations with different variables can silently collapse into one. Use include_role in tasks for conditional composition.

One baseline play, many function plays

When a host belongs to several functional groups and each group's playbook lists the same base role, the base role runs once per group per apply -- role dedup only applies within a single play. Structure the top-level playbook as one hosts: all baseline play followed by function-specific plays that do not re-list the base:

# site.yml
- hosts: all
  roles: [base]

- hosts: webservers
  roles: [web_app]    # no base here; it ran above

- hosts: dbservers
  roles: [db_app]

Tradeoff: running a single function playbook standalone no longer implicitly baselines. Handle it with documentation or a wrapper recipe. Avoid workarounds like play-scoped set_fact guards, run_once, or tag gating -- they paper over a structural issue.

Role variable surface

A role consumes variables from two structurally different places:

• Role-owned variables (prefixed with the role name, e.g. myapp_port) live in defaults/main.yml with sensible defaults. This file is the source of truth for what the role can be tuned with.
• Upstream variables (values that come from group_vars, host_vars, or another role's output) are declared in meta/argument_specs.yml with a type, required: true (or required: false plus default: when the role must stay safe with the variable unset), and choices where applicable. They are NOT mirrored into defaults/main.yml.

Ansible validates argument_specs before tasks/main.yml runs, so a missing required variable or a wrong-typed value fails fast at the top of the role rather than surfacing as an obscure Jinja traceback.

# roles/consumer/meta/argument_specs.yml
---
argument_specs:
  main:
    short_description: Consume an upstream service port.
    options:
      upstream_service_port:
        description: TCP port the upstream service listens on.
        type: int
        required: true

Disjoint-keyset rule. defaults/main.yml and meta/argument_specs.yml must have disjoint key sets -- a variable is either role-owned or upstream, never both. Drift between the two is avoided by convention, not tooling. Treat a variable appearing in both as a review-time bug. The ansible-lint role-argument-spec rule requires every role to ship a meta/argument_specs.yml, even if options: is empty.

Circular dependencies

If role A and role B depend on each other, do not resolve with mutual meta deps. Extract the shared concern into a third role C and have both depend on C. If the "cycle" is actually a sequencing requirement (A produces, B consumes, A uses B's output), split the work across ordered plays rather than forcing it into role metadata -- common with CAs, service discovery, and secrets managers.

See references/role-composition.md for the full patterns, Ansible docs citations, argument_specs depth (description style, what it catches, vault indirection), and the anti-pattern breakdown.

Idempotency and Best Practices

Every task must be idempotent: running it N times produces the same result as running it once, with changed=0 on subsequent runs.

Use modules, not shell commands

# BAD
- name: Install nginx
  ansible.builtin.shell: apt-get install -y nginx

# GOOD
- name: Install nginx
  ansible.builtin.apt:
    name: nginx
    state: present

When shell/command is unavoidable, always set changed_when and prefer command over shell (no shell injection surface). Guard interpolated variables with the quote filter:

# DANGEROUS
- ansible.builtin.shell: "grep {{ username }} /etc/passwd"

# SAFE
- ansible.builtin.shell: "grep {{ username | quote }} /etc/passwd"

# BEST -- use creates/removes guards
- name: Run database migration
  ansible.builtin.command:
    cmd: /opt/app/bin/migrate --up
    creates: /opt/app/.migrated
  register: migrate_result
  changed_when: "'Applied' in migrate_result.stdout"

Notify handlers instead of inline restarts

# BAD -- restarts every run
- name: Deploy config
  ansible.builtin.template:
    src: app.conf.j2
    dest: /etc/app/app.conf
- name: Restart app
  ansible.builtin.systemd:
    name: myapp
    state: restarted

# GOOD -- restart only when config changes
- name: Deploy config
  ansible.builtin.template:
    src: app.conf.j2
    dest: /etc/app/app.conf
  notify: Restart app

Set become per-task, not per-play

# BAD -- everything runs as root
- hosts: webservers
  become: true
  tasks:
    - name: Read status (does NOT need root)
      ansible.builtin.command: /opt/app/status
      changed_when: false

# GOOD -- least privilege
- hosts: webservers
  tasks:
    - name: Read status
      ansible.builtin.command: /opt/app/status
      changed_when: false
    - name: Restart nginx (needs root)
      ansible.builtin.systemd:
        name: nginx
        state: restarted
      become: true

Common idempotency breakers

Verify idempotency with Molecule

Use molecule converge && molecule idempotence to iterate quickly. molecule test runs the full lifecycle including the idempotence step. For tasks that legitimately cannot be idempotent, tag with molecule-idempotence-notest. See references/testing.md.

Common Gotchas

{{ }} in when: -- when: already evaluates Jinja. Double-wrapping causes errors:

# BAD
when: "{{ my_var == 'foo' }}"
# GOOD
when: my_var == 'foo'

Bare booleans in YAML -- Unquoted yes, no, on, off are parsed as booleans, not strings:

# BAD -- 'on' becomes boolean True
feature_flags:
  on: enabled
# GOOD
feature_flags:
  "on": enabled

collections: keyword -- Do not use collections: at play level. It breaks portability, hides module origins, and ansible-lint rejects it (fqcn[keyword]). Always use full FQCN.

set_fact persists for the host -- Facts survive across plays in the same run. A set_fact in play 1 can override an inventory variable in play 2. Prefer vars: at play/task level when values do not need to persist.

delegate_to + become -- become applies on the delegated host. A task delegated to localhost with become: true escalates on localhost, which may be unintended.

Dictionary merging -- Ansible replaces dictionaries entirely rather than merging. If group_vars/all defines app_config: {port: 80, workers: 4} and a host overrides app_config: {port: 9090}, workers is gone. Use flat variables or the combine filter.

register on skipped tasks -- A registered variable from a skipped task has .skipped == true but no .stdout. Always check result is not skipped before accessing attributes.

Jinja2 Templating

Common Filters

Whitespace Control

Jinja2 tags produce blank lines in rendered output. Use dash-trimming to remove them:

# Without control -- leaves blank lines
{% if enable_ssl %}
ssl_certificate /etc/ssl/cert.pem;
{% endif %}

# With control -- clean output
{%- if enable_ssl %}
ssl_certificate /etc/ssl/cert.pem;
{%- endif %}

{%- trims whitespace before the tag. -%} trims whitespace after the tag. Use on control-flow tags (if, for, endif, endfor), not on expression tags ({{ }}).

The omit Sentinel

Use omit with default to conditionally exclude a module parameter entirely, as if it were never specified:

- name: Create user with optional groups
  ansible.builtin.user:
    name: "{{ username }}"
    groups: "{{ user_groups | default(omit) }}"

When user_groups is undefined, the groups parameter is omitted from the module call and the module uses its own default behavior. This is different from passing an empty string or None.

Security Essentials

Vault indirection pattern

# group_vars/production/vars.yml (plaintext, grep-able)
db_password: "{{ vault_db_password }}"

# group_vars/production/vault.yml (encrypted: ansible-vault encrypt)
vault_db_password: "s3cret"

ansible-vault rekey works on file-level encryption only, NOT on inline encrypt_string values. For frequently rotated secrets, use an external secrets manager.

no_log on sensitive tasks

- name: Set database password
  ansible.builtin.user:
    name: dbuser
    password: "{{ db_password | password_hash('sha512') }}"
  no_log: true

no_log does not prevent exposure in -vvvv tracebacks, some callback plugins, or loop results on older versions. Treat it as defense-in-depth, not sole protection.

ansible.cfg security

Never run Ansible from untrusted directories -- Ansible loads ansible.cfg from the current directory. Set ANSIBLE_CONFIG explicitly in CI. Keep host_key_checking = True (the default).

See references/security.md for Vault IDs, external secrets managers, SSH hardening, audit logging, and supply chain security.

Molecule Testing

Molecule tests roles in ephemeral environments. Use the ansible-native approach (Molecule 6+) where create/destroy are standard Ansible playbooks.

Minimal role test setup

roles/myrole/molecule/default/
  molecule.yml
  converge.yml
  verify.yml

# molecule.yml
---
dependency:
  name: galaxy
  options:
    requirements-file: requirements.yml

# converge.yml
---
- name: Converge
  hosts: all
  tasks:
    - name: Include the role
      ansible.builtin.include_role:
        name: myrole

# verify.yml
---
- name: Verify
  hosts: all
  gather_facts: true
  tasks:
    - name: Gather package facts
      ansible.builtin.package_facts:

    - name: Assert nginx is installed
      ansible.builtin.assert:
        that: "'nginx' in ansible_facts.packages"

    - name: Check nginx is running
      ansible.builtin.service_facts:

    - name: Assert nginx service active
      ansible.builtin.assert:
        that: ansible_facts.services['nginx.service'].state == 'running'

Development workflow: molecule converge (iterate), molecule idempotence (confirm no changes on re-run), molecule test (full lifecycle).

See references/testing.md for multi-platform testing, custom create/destroy playbooks, CI/CD integration, and advanced scenarios.

Performance

• Forks: Increase forks in ansible.cfg (default 5). Set to ~20-50 for large inventories.
• Pipelining: Set pipelining = True under [ssh_connection]. Requires Defaults !requiretty in sudoers.
• Fact caching: fact_caching = jsonfile with fact_caching_connection = /tmp/ansible_facts.
• gather_subset: Use gather_subset: [min] or gather_facts: false when full facts are not needed.
• Strategy: Use strategy: free to let fast hosts proceed independently (default linear waits for all hosts per task).
• Async: For long tasks, async: 3600 + poll: 0 for fire-and-forget, then check with async_status.

Validation Pipeline

yamllint .                                      # YAML syntax
ansible-lint                                    # Ansible best practices
ansible-playbook site.yml --syntax-check        # playbook structure
ansible-playbook site.yml --check --diff        # dry run
molecule test                                   # integration test