Deployment Troubleshooting Guide
When a deployment fails, the error message is your best starting point — but Terraform and cloud provider error messages are often cryptic or misleading. This guide maps common error patterns to their root causes and fixes, organized by the deployment phase where they typically occur.
When helping a user debug a deployment issue, start by identifying which phase failed, then match the error text against the patterns below.
Phase 1: Terraform Init Errors
Init failures happen before any infrastructure is created. They're usually about provider plugins or backend configuration.
| Error Pattern |
Root Cause |
Fix |
Failed to install provider |
No internet, proxy blocking registry.terraform.io, or provider name typo |
Check connectivity: curl -I https://registry.terraform.io. If behind proxy, set HTTPS_PROXY. Verify provider source string |
Could not load plugin |
Plugin cache corrupted or provider version mismatch |
Run terraform init -upgrade to re-download. Delete .terraform/ and retry if persistent |
Backend initialization required |
Remote state bucket doesn't exist or credentials wrong |
Create the bucket first, verify credentials have access to it. Check region matches |
Failed to query available provider packages |
DNS resolution failure or firewall blocking |
Try nslookup registry.terraform.io. Consider using terraform init -plugin-dir with pre-downloaded providers |
Phase 2: Authentication Errors
These surface during terraform plan when the provider tries to validate credentials against the cloud API.
| Error Pattern |
Provider |
Fix |
NoCredentialProviders |
AWS |
AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY not set or expired. Re-export them or check RedC profile |
AuthorizationFailed |
Azure |
Service principal lacks permissions on the subscription. Verify ARM_SUBSCRIPTION_ID matches, check role assignments |
googleapi: Error 403 |
GCP |
Service account doesn't have required permissions. Check IAM roles in GCP Console, verify GOOGLE_APPLICATION_CREDENTIALS path |
InvalidAccessKeyId |
AWS/Alibaba |
Access key deleted or rotated. Generate a new key pair in the console |
AuthFailure |
Tencent |
TENCENTCLOUD_SECRET_ID wrong (note: it's "SecretId" not "AccessKey") |
Debugging tip: When a user reports an auth error, ask them to verify the environment variables are set in the current shell session. A common mistake is setting them in one terminal and running RedC in another.
Phase 3: Resource Creation Failures
These happen during terraform apply when the cloud provider rejects a resource creation request.
| Error Pattern |
Root Cause |
Fix |
InstanceLimitExceeded |
Account quota reached for this instance type |
Request a quota increase via support ticket, or use a different instance type/region |
VPCLimitExceeded |
Default limit is 5 VPCs per region |
Clean up unused VPCs in the console, or request a limit increase |
InvalidParameterValue for instance type |
Instance type not available in the selected AZ |
Check availability with aws ec2 describe-instance-type-offerings, try a different AZ or type |
InsufficientInstanceCapacity |
AWS capacity constraints in that AZ |
Retry in a different AZ (-a, -b, -c), or try a different instance family |
Insufficient balance |
Prepaid account ran out of credit |
Top up the account. Use get_balances to check current balance |
Phase 4: Network and Connectivity Issues
These typically appear after instances are created but the user can't reach them.
| Symptom |
Likely Cause |
Investigation Steps |
| SSH connection refused |
Security group doesn't allow inbound SSH from user's IP |
Check the security group ingress rules. Verify user's current public IP matches the allowed CIDR |
| SSH connection timed out |
Instance has no public IP, or is in a private subnet without NAT |
Verify the instance has a public IP in the console. Check subnet route table has an internet gateway |
timeout awaiting response during apply |
Security group blocks outbound HTTPS (443) |
The instance needs outbound access to download packages. Check egress rules |
| Instance created but tools don't work |
user_data script failed silently |
SSH in and check /var/log/cloud-init-output.log for errors |
Phase 5: State Issues
State problems are dangerous because they can cause Terraform to lose track of real infrastructure, leading to orphaned resources you're still paying for.
| Error Pattern |
Root Cause |
Fix |
Error acquiring the state lock |
Another terraform apply is running, or a previous run crashed without releasing the lock |
Wait for the other process to finish. If it crashed, force-unlock: terraform force-unlock <LOCK_ID> |
Resource already exists |
Resource was created outside Terraform (e.g., manually in console) |
Import it: terraform import <resource_address> <resource_id> |
Unsupported attribute |
Provider version upgraded and the attribute name changed |
Pin provider version in required_providers, or update your .tf to use the new attribute name |
| Drift between state and reality |
Manual changes in cloud console |
Run terraform plan to see the diff, then decide: apply to overwrite manual changes, or terraform refresh to update state |
Phase 6: User Data and Provisioning
Cloud-init runs on first boot and its failures are silent from Terraform's perspective — the instance is "created" but not properly configured.
| Symptom |
Investigation |
Fix |
| Packages not installed |
Check /var/log/cloud-init-output.log |
Usually DNS or proxy issues. Add apt update retry logic to the script |
| Script didn't run at all |
Check /var/log/cloud-init.log for YAML parse errors |
Validate the cloud-init YAML syntax. Common issue: wrong indentation in write_files |
| Script timed out |
Long-running operations (compiling, large downloads) |
Break into smaller scripts, or increase timeout. Consider using RedC's exec_command for post-deploy setup instead |
| Wrong permissions on files |
write_files defaults to root ownership |
Set owner and permissions explicitly in the cloud-init config |
1---2name: deployment-troubleshooting3description: Diagnose and fix Terraform deployment errors in RedC scenarios. Use this skill whenever the user encounters an error during deployment — whether it's a Terraform init failure, authentication error, resource creation failure, network timeout, state conflict, or cloud-init problem. Also use when the user pastes an error message, says "deployment failed", asks why something isn't working, or reports that instances are unreachable after creation. This skill covers the most common failure modes across all cloud providers supported by RedC.4---56# Deployment Troubleshooting Guide78When a deployment fails, the error message is your best starting point — but Terraform and cloud provider error messages are often cryptic or misleading. This guide maps common error patterns to their root causes and fixes, organized by the deployment phase where they typically occur.910When helping a user debug a deployment issue, start by identifying which phase failed, then match the error text against the patterns below.1112## Phase 1: Terraform Init Errors1314Init failures happen before any infrastructure is created. They're usually about provider plugins or backend configuration.1516| Error Pattern | Root Cause | Fix |17|---------------|-----------|-----|18| `Failed to install provider` | No internet, proxy blocking registry.terraform.io, or provider name typo | Check connectivity: `curl -I https://registry.terraform.io`. If behind proxy, set `HTTPS_PROXY`. Verify provider source string |19| `Could not load plugin` | Plugin cache corrupted or provider version mismatch | Run `terraform init -upgrade` to re-download. Delete `.terraform/` and retry if persistent |20| `Backend initialization required` | Remote state bucket doesn't exist or credentials wrong | Create the bucket first, verify credentials have access to it. Check region matches |21| `Failed to query available provider packages` | DNS resolution failure or firewall blocking | Try `nslookup registry.terraform.io`. Consider using `terraform init -plugin-dir` with pre-downloaded providers |2223## Phase 2: Authentication Errors2425These surface during `terraform plan` when the provider tries to validate credentials against the cloud API.2627| Error Pattern | Provider | Fix |28|---------------|----------|-----|29| `NoCredentialProviders` | AWS | `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` not set or expired. Re-export them or check RedC profile |30| `AuthorizationFailed` | Azure | Service principal lacks permissions on the subscription. Verify `ARM_SUBSCRIPTION_ID` matches, check role assignments |31| `googleapi: Error 403` | GCP | Service account doesn't have required permissions. Check IAM roles in GCP Console, verify `GOOGLE_APPLICATION_CREDENTIALS` path |32| `InvalidAccessKeyId` | AWS/Alibaba | Access key deleted or rotated. Generate a new key pair in the console |33| `AuthFailure` | Tencent | `TENCENTCLOUD_SECRET_ID` wrong (note: it's "SecretId" not "AccessKey") |3435**Debugging tip:** When a user reports an auth error, ask them to verify the environment variables are set in the current shell session. A common mistake is setting them in one terminal and running RedC in another.3637## Phase 3: Resource Creation Failures3839These happen during `terraform apply` when the cloud provider rejects a resource creation request.4041| Error Pattern | Root Cause | Fix |42|---------------|-----------|-----|43| `InstanceLimitExceeded` | Account quota reached for this instance type | Request a quota increase via support ticket, or use a different instance type/region |44| `VPCLimitExceeded` | Default limit is 5 VPCs per region | Clean up unused VPCs in the console, or request a limit increase |45| `InvalidParameterValue` for instance type | Instance type not available in the selected AZ | Check availability with `aws ec2 describe-instance-type-offerings`, try a different AZ or type |46| `InsufficientInstanceCapacity` | AWS capacity constraints in that AZ | Retry in a different AZ (`-a`, `-b`, `-c`), or try a different instance family |47| `Insufficient balance` | Prepaid account ran out of credit | Top up the account. Use `get_balances` to check current balance |4849## Phase 4: Network and Connectivity Issues5051These typically appear after instances are created but the user can't reach them.5253| Symptom | Likely Cause | Investigation Steps |54|---------|-------------|---------------------|55| SSH connection refused | Security group doesn't allow inbound SSH from user's IP | Check the security group ingress rules. Verify user's current public IP matches the allowed CIDR |56| SSH connection timed out | Instance has no public IP, or is in a private subnet without NAT | Verify the instance has a public IP in the console. Check subnet route table has an internet gateway |57| `timeout awaiting response` during apply | Security group blocks outbound HTTPS (443) | The instance needs outbound access to download packages. Check egress rules |58| Instance created but tools don't work | user_data script failed silently | SSH in and check `/var/log/cloud-init-output.log` for errors |5960## Phase 5: State Issues6162State problems are dangerous because they can cause Terraform to lose track of real infrastructure, leading to orphaned resources you're still paying for.6364| Error Pattern | Root Cause | Fix |65|---------------|-----------|-----|66| `Error acquiring the state lock` | Another `terraform apply` is running, or a previous run crashed without releasing the lock | Wait for the other process to finish. If it crashed, force-unlock: `terraform force-unlock <LOCK_ID>` |67| `Resource already exists` | Resource was created outside Terraform (e.g., manually in console) | Import it: `terraform import <resource_address> <resource_id>` |68| `Unsupported attribute` | Provider version upgraded and the attribute name changed | Pin provider version in `required_providers`, or update your `.tf` to use the new attribute name |69| Drift between state and reality | Manual changes in cloud console | Run `terraform plan` to see the diff, then decide: apply to overwrite manual changes, or `terraform refresh` to update state |7071## Phase 6: User Data and Provisioning7273Cloud-init runs on first boot and its failures are silent from Terraform's perspective — the instance is "created" but not properly configured.7475| Symptom | Investigation | Fix |76|---------|---------------|-----|77| Packages not installed | Check `/var/log/cloud-init-output.log` | Usually DNS or proxy issues. Add `apt update` retry logic to the script |78| Script didn't run at all | Check `/var/log/cloud-init.log` for YAML parse errors | Validate the cloud-init YAML syntax. Common issue: wrong indentation in `write_files` |79| Script timed out | Long-running operations (compiling, large downloads) | Break into smaller scripts, or increase timeout. Consider using RedC's `exec_command` for post-deploy setup instead |80| Wrong permissions on files | `write_files` defaults to root ownership | Set `owner` and `permissions` explicitly in the cloud-init config |
wgpsec