micsapp Detecting Infrastructure Drift
Overview
Detect discrepancies between actual cloud infrastructure state and the desired state defined in IaC (Terraform, CloudFormation, Pulumi). Run drift detection commands, analyze modified/added/deleted resources, generate drift reports with affected resources, and provide remediation steps to bring infrastructure back into compliance.
Prerequisites
- IaC configuration files up to date in the project directory
- Cloud provider CLI installed and authenticated with read access to all managed resources
- IaC tool installed: Terraform 1.0+, AWS CLI (for CloudFormation drift), or Pulumi
- Remote state storage accessible and current (S3 backend, Terraform Cloud, Pulumi Cloud)
- Read-only IAM permissions for all resource types managed by IaC
Instructions
- Identify the IaC tool in use by scanning for
.tffiles,template.yaml, orPulumi.yaml - Initialize the IaC tool if needed:
terraform initto download providers and configure backend - Run drift detection:
terraform plan -detailed-exitcode(exit code 2 = drift detected),aws cloudformation detect-stack-drift, orpulumi preview - Parse the output to identify resources with drift: added (exists in cloud but not in IaC), modified (attributes changed), or deleted (in IaC but missing from cloud)
- For each drifted resource, determine if the drift is intentional (manual hotfix) or unintentional (configuration error, unauthorized change)
- Generate a structured drift report with resource identifiers, attribute differences, and severity classification
- Provide remediation options per resource:
terraform applyto enforce desired state,terraform importto adopt changes, or update IaC to match reality - Schedule recurring drift detection: configure a cron job or CI pipeline to run daily and alert on drift
- Investigate the root cause: determine who made the manual change and implement guardrails (SCPs, IAM restrictions) to prevent recurrence
Output
- Drift detection report with resource-level detail: resource type, ID, drifted attributes, expected vs. actual values
- Remediation commands:
terraform apply,terraform import, or IaC code updates - CI/CD pipeline step for automated drift detection on a schedule
- Alert configuration for drift detection results (Slack, email, PagerDuty)
- Prevention recommendations: IAM policy restrictions, SCP guardrails, automated enforcement
Error Handling
| Error | Cause | Solution |
|---|---|---|
Error acquiring state lock |
Another Terraform process is running or stale lock | Wait for the other process; use terraform force-unlock <ID> if the lock is stale |
Unable to authenticate to cloud provider |
Expired or missing credentials | Refresh with aws configure, gcloud auth login, or az login |
No state file found |
Backend not initialized or state file deleted | Run terraform init to configure the backend; restore state from backup if deleted |
Access denied reading resource |
IAM policy missing read permissions for some resource types | Grant read-only access for all resource types managed by IaC (ReadOnlyAccess or specific policies) |
State file version mismatch |
Terraform version newer than state format | Upgrade Terraform to match the state version or use terraform state replace-provider |
Examples
- "Run drift detection against all Terraform-managed infrastructure and generate a report of resources that have changed since last apply."
- "Set up a daily GitHub Actions workflow that runs
terraform planand posts drift results to Slack if any resources are out of sync." - "Detect CloudFormation stack drift for the production VPC stack and provide remediation steps for any MODIFIED resources."
Resources
- Terraform drift detection: https://developer.hashicorp.com/terraform/tutorials/state/resource-drift
- CloudFormation drift detection: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/detect-drift-stack.html
- Pulumi drift detection: https://www.pulumi.com/docs/using-pulumi/pulumi-packages/guides/drift-detection/
- Preventing drift: https://developer.hashicorp.com/terraform/tutorials/state/refresh