AWS Cost Analyzer

AWS Cost Analyzer Skill

Overview

AWS cost analysis and optimization skill for cloud infrastructure. Analyzes costs across services (EC2, RDS, S3, Lambda), identifies unused resources, provides right-sizing recommendations, and optimizes Reserved Instances.

Capabilities

1. Cost Analysis

• Cost breakdown by service
• Cost trends over time
• Cost allocation by project/tag
• Budget vs. actual spending
• Forecasting

2. Resource Optimization

• Unused resource identification
• Right-sizing recommendations
• Reserved Instance optimization
• Savings Plan analysis
• Spot Instance opportunities

3. Storage Optimization

• S3 lifecycle policies
• EBS volume optimization
• Snapshot management
• Intelligent-Tiering setup

4. Monitoring & Alerts

• Cost anomaly detection
• Budget alerts
• Resource utilization tracking
• Cost allocation tags

AWS Cost Explorer CLI

Setup

# Install AWS CLI
brew install awscli  # macOS
# Or: pip install awscli

# Configure credentials
aws configure

# Install jq for JSON parsing
brew install jq

Basic Cost Queries

# Get total costs for last month
aws ce get-cost-and-usage \
  --time-period Start=$(date -d "1 month ago" +%Y-%m-01),End=$(date +%Y-%m-01) \
  --granularity MONTHLY \
  --metrics BlendedCost \
  | jq '.ResultsByTime[].Total.BlendedCost'

# Costs by service (last 30 days)
aws ce get-cost-and-usage \
  --time-period Start=$(date -d "30 days ago" +%Y-%m-%d),End=$(date +%Y-%m-%d) \
  --granularity MONTHLY \
  --metrics BlendedCost \
  --group-by Type=DIMENSION,Key=SERVICE \
  | jq '.ResultsByTime[].Groups[] | {service: .Keys[0], cost: .Metrics.BlendedCost.Amount}'

# Costs by project tag
aws ce get-cost-and-usage \
  --time-period Start=$(date -d "30 days ago" +%Y-%m-%d),End=$(date +%Y-%m-%d) \
  --granularity MONTHLY \
  --metrics BlendedCost \
  --group-by Type=TAG,Key=Project \
  | jq '.ResultsByTime[].Groups[]'

Cost Optimization Strategies

1. EC2 Right-Sizing

Identify Underutilized Instances:

# Get EC2 instances with CPU utilization < 10%
aws cloudwatch get-metric-statistics \
  --namespace AWS/EC2 \
  --metric-name CPUUtilization \
  --dimensions Name=InstanceId,Value=i-1234567890abcdef0 \
  --start-time $(date -u -d '7 days ago' +%Y-%m-%dT%H:%M:%S) \
  --end-time $(date -u +%Y-%m-%dT%H:%M:%S) \
  --period 86400 \
  --statistics Average \
  | jq '.Datapoints[] | select(.Average < 10)'

Right-Sizing Recommendations:

# Example: t3.xlarge → t3.large (save ~50%)
# Current: 4 vCPU, 16 GB RAM, $0.1664/hour
# Recommended: 2 vCPU, 8 GB RAM, $0.0832/hour
# Savings: ~$60/month per instance

2. Reserved Instances & Savings Plans

Calculate Potential Savings:

# Get RI recommendations
aws ce get-reservation-purchase-recommendation \
  --service "Amazon Elastic Compute Cloud - Compute" \
  --lookback-period-in-days SIXTY_DAYS \
  --payment-option NO_UPFRONT \
  | jq '.Recommendations[] | {
      instanceType: .RecommendationDetails.AmazonEC2.InstanceType,
      monthlySavings: .RecommendationDetails.EstimatedMonthlySavings,
      upfrontCost: .RecommendationDetails.UpfrontCost
    }'

Typical Savings:

• 1-year RI, No Upfront: ~40% savings
• 3-year RI, All Upfront: ~60% savings
• Compute Savings Plan: ~66% savings

3. S3 Storage Optimization

Lifecycle Policies:

{
  "Rules": [
    {
      "Id": "archive-old-logs",
      "Status": "Enabled",
      "Filter": {
        "Prefix": "logs/"
      },
      "Transitions": [
        {
          "Days": 30,
          "StorageClass": "STANDARD_IA"
        },
        {
          "Days": 90,
          "StorageClass": "GLACIER"
        }
      ],
      "Expiration": {
        "Days": 365
      }
    },
    {
      "Id": "intelligent-tiering",
      "Status": "Enabled",
      "Filter": {
        "Prefix": "data/"
      },
      "Transitions": [
        {
          "Days": 0,
          "StorageClass": "INTELLIGENT_TIERING"
        }
      ]
    }
  ]
}

Apply Lifecycle Policy:

aws s3api put-bucket-lifecycle-configuration \
  --bucket siae-dpi-bucket \
  --lifecycle-configuration file://lifecycle-policy.json

Storage Cost Comparison:

4. Database (RDS) Optimization

Identify Idle RDS Instances:

# Check database connections
aws cloudwatch get-metric-statistics \
  --namespace AWS/RDS \
  --metric-name DatabaseConnections \
  --dimensions Name=DBInstanceIdentifier,Value=siae-publishing-db \
  --start-time $(date -u -d '7 days ago' +%Y-%m-%dT%H:%M:%S) \
  --end-time $(date -u +%Y-%m-%dT%H:%M:%S) \
  --period 86400 \
  --statistics Average

Right-Sizing RDS:

# CPU utilization
aws cloudwatch get-metric-statistics \
  --namespace AWS/RDS \
  --metric-name CPUUtilization \
  --dimensions Name=DBInstanceIdentifier,Value=siae-tunex-db \
  --start-time $(date -u -d '30 days ago' +%Y-%m-%dT%H:%M:%S) \
  --end-time $(date -u +%Y-%m-%dT%H:%M:%S) \
  --period 86400 \
  --statistics Average Maximum

# If Average < 20% and Maximum < 60%, consider downsizing

5. Lambda Optimization

Memory Right-Sizing:

# Current: 1024 MB, Average usage: 400 MB
# Recommendation: 512 MB
# Savings: ~50% cost reduction

Analyze Lambda Performance:

# Get Lambda invocations and duration
aws cloudwatch get-metric-statistics \
  --namespace AWS/Lambda \
  --metric-name Duration \
  --dimensions Name=FunctionName,Value=siae-dpi-processor \
  --start-time $(date -u -d '7 days ago' +%Y-%m-%dT%H:%M:%S) \
  --end-time $(date -u +%Y-%m-%dT%H:%M:%S) \
  --period 3600 \
  --statistics Average Maximum

Integration Scripts

aws_cost_report.py

Generate comprehensive cost report:

#!/usr/bin/env python3
import boto3
import json
from datetime import datetime, timedelta

def get_monthly_costs():
    """Get costs for current month by service"""
    ce = boto3.client('ce', region_name='us-east-1')

    start_date = datetime.now().replace(day=1).strftime('%Y-%m-%d')
    end_date = datetime.now().strftime('%Y-%m-%d')

    response = ce.get_cost_and_usage(
        TimePeriod={'Start': start_date, 'End': end_date},
        Granularity='MONTHLY',
        Metrics=['BlendedCost'],
        GroupBy=[{'Type': 'DIMENSION', 'Key': 'SERVICE'}]
    )

    costs = []
    for result in response['ResultsByTime']:
        for group in result['Groups']:
            service = group['Keys'][0]
            cost = float(group['Metrics']['BlendedCost']['Amount'])
            if cost > 0:
                costs.append({'service': service, 'cost': cost})

    return sorted(costs, key=lambda x: x['cost'], reverse=True)

def get_project_costs():
    """Get costs by project tag"""
    ce = boto3.client('ce', region_name='us-east-1')

    start_date = datetime.now().replace(day=1).strftime('%Y-%m-%d')
    end_date = datetime.now().strftime('%Y-%m-%d')

    response = ce.get_cost_and_usage(
        TimePeriod={'Start': start_date, 'End': end_date},
        Granularity='MONTHLY',
        Metrics=['BlendedCost'],
        GroupBy=[{'Type': 'TAG', 'Key': 'Project'}]
    )

    costs = []
    for result in response['ResultsByTime']:
        for group in result['Groups']:
            project = group['Keys'][0].split('$')[1] if '$' in group['Keys'][0] else 'Untagged'
            cost = float(group['Metrics']['BlendedCost']['Amount'])
            if cost > 0:
                costs.append({'project': project, 'cost': cost})

    return sorted(costs, key=lambda x: x['cost'], reverse=True)

def print_report():
    """Print formatted cost report"""
    print("=== AWS Cost Report ===\n")
    print(f"Period: {datetime.now().strftime('%B %Y')}\n")

    # Service breakdown
    print("💰 Costs by Service:")
    service_costs = get_monthly_costs()
    total = sum(c['cost'] for c in service_costs)

    for cost_data in service_costs[:10]:
        service = cost_data['service']
        cost = cost_data['cost']
        percentage = (cost / total * 100) if total > 0 else 0
        print(f"  {service:40} ${cost:10.2f} ({percentage:5.1f}%)")

    print(f"\n  {'Total':40} ${total:10.2f}\n")

    # Project breakdown
    print("📊 Costs by Project:")
    project_costs = get_project_costs()

    for cost_data in project_costs:
        project = cost_data['project']
        cost = cost_data['cost']
        percentage = (cost / total * 100) if total > 0 else 0
        print(f"  {project:40} ${cost:10.2f} ({percentage:5.1f}%)")

    print("\n=== Top Optimization Opportunities ===")
    print("1. Review EC2 instances with <20% CPU utilization")
    print("2. Implement S3 lifecycle policies for old data")
    print("3. Consider Reserved Instances for steady workloads")
    print("4. Enable S3 Intelligent-Tiering for unknown patterns")
    print("5. Rightsize RDS instances based on metrics")

if __name__ == '__main__':
    print_report()

find_unused_resources.py

Identify unused AWS resources:

#!/usr/bin/env python3
import boto3
from datetime import datetime, timedelta

def find_unused_ebs_volumes():
    """Find unattached EBS volumes"""
    ec2 = boto3.client('ec2')
    volumes = ec2.describe_volumes(
        Filters=[{'Name': 'status', 'Values': ['available']}]
    )

    unused = []
    for vol in volumes['Volumes']:
        size = vol['Size']
        vol_type = vol['VolumeType']
        cost_per_gb = 0.10 if vol_type == 'gp3' else 0.10  # Approximate

        unused.append({
            'id': vol['VolumeId'],
            'size': size,
            'type': vol_type,
            'monthly_cost': size * cost_per_gb
        })

    return unused

def find_unused_elastic_ips():
    """Find unassociated Elastic IPs"""
    ec2 = boto3.client('ec2')
    addresses = ec2.describe_addresses()

    unused = []
    for addr in addresses['Addresses']:
        if 'InstanceId' not in addr:
            unused.append({
                'ip': addr['PublicIp'],
                'allocation_id': addr['AllocationId'],
                'monthly_cost': 3.60  # $0.005/hour for unused EIP
            })

    return unused

def find_old_snapshots():
    """Find old EBS snapshots (>90 days)"""
    ec2 = boto3.client('ec2')
    snapshots = ec2.describe_snapshots(OwnerIds=['self'])

    old_snapshots = []
    cutoff_date = datetime.now() - timedelta(days=90)

    for snap in snapshots['Snapshots']:
        start_time = snap['StartTime'].replace(tzinfo=None)
        if start_time < cutoff_date:
            old_snapshots.append({
                'id': snap['SnapshotId'],
                'size': snap['VolumeSize'],
                'age_days': (datetime.now() - start_time).days,
                'monthly_cost': snap['VolumeSize'] * 0.05
            })

    return old_snapshots

def generate_cleanup_report():
    """Generate resource cleanup report"""
    print("=== Unused AWS Resources Report ===\n")

    # EBS Volumes
    print("💾 Unattached EBS Volumes:")
    volumes = find_unused_ebs_volumes()
    if volumes:
        for vol in volumes:
            print(f"  {vol['id']}: {vol['size']}GB {vol['type']} - ${vol['monthly_cost']:.2f}/month")
        total_vol_cost = sum(v['monthly_cost'] for v in volumes)
        print(f"  Total potential savings: ${total_vol_cost:.2f}/month\n")
    else:
        print("  ✅ No unused volumes found\n")

    # Elastic IPs
    print("🌐 Unassociated Elastic IPs:")
    eips = find_unused_elastic_ips()
    if eips:
        for eip in eips:
            print(f"  {eip['ip']} ({eip['allocation_id']}) - ${eip['monthly_cost']:.2f}/month")
        total_eip_cost = sum(e['monthly_cost'] for e in eips)
        print(f"  Total potential savings: ${total_eip_cost:.2f}/month\n")
    else:
        print("  ✅ No unused EIPs found\n")

    # Old Snapshots
    print("📸 Old EBS Snapshots (>90 days):")
    snapshots = find_old_snapshots()
    if snapshots:
        for snap in snapshots[:10]:
            print(f"  {snap['id']}: {snap['size']}GB, {snap['age_days']} days old - ${snap['monthly_cost']:.2f}/month")
        if len(snapshots) > 10:
            print(f"  ... and {len(snapshots) - 10} more")
        total_snap_cost = sum(s['monthly_cost'] for s in snapshots)
        print(f"  Total potential savings: ${total_snap_cost:.2f}/month\n")
    else:
        print("  ✅ No old snapshots found\n")

    # Summary
    total_savings = (
        sum(v['monthly_cost'] for v in volumes) +
        sum(e['monthly_cost'] for e in eips) +
        sum(s['monthly_cost'] for s in snapshots)
    )

    print(f"=== Total Potential Monthly Savings: ${total_savings:.2f} ===")
    print(f"=== Annual Savings: ${total_savings * 12:.2f} ===")

if __name__ == '__main__':
    generate_cleanup_report()

rightsize_recommendations.sh

Generate EC2 right-sizing recommendations:

#!/bin/bash
# EC2 right-sizing recommendations

echo "=== EC2 Right-Sizing Analysis ==="
echo

# Get all running instances
aws ec2 describe-instances \
  --filters "Name=instance-state-name,Values=running" \
  --query 'Reservations[].Instances[].[InstanceId,InstanceType,Tags[?Key==`Name`].Value|[0]]' \
  --output text | while read instance_id instance_type name; do

    echo "Analyzing: $name ($instance_id - $instance_type)"

    # Get average CPU utilization (last 7 days)
    cpu_avg=$(aws cloudwatch get-metric-statistics \
      --namespace AWS/EC2 \
      --metric-name CPUUtilization \
      --dimensions Name=InstanceId,Value=$instance_id \
      --start-time $(date -u -d '7 days ago' +%Y-%m-%dT%H:%M:%S) \
      --end-time $(date -u +%Y-%m-%dT%H:%M:%S) \
      --period 86400 \
      --statistics Average \
      --query 'Datapoints[].Average' \
      --output text | awk '{sum+=$1; count++} END {if(count>0) print sum/count; else print 0}')

    # Recommendation logic
    if (( $(echo "$cpu_avg < 10" | bc -l) )); then
        echo "  ⚠️  CPU: ${cpu_avg}% - Consider stopping or downsizing"
    elif (( $(echo "$cpu_avg < 30" | bc -l) )); then
        echo "  ⚠️  CPU: ${cpu_avg}% - Consider downsizing"
    elif (( $(echo "$cpu_avg > 80" | bc -l) )); then
        echo "  ⚠️  CPU: ${cpu_avg}% - Consider upsizing"
    else
        echo "  ✅ CPU: ${cpu_avg}% - Well-sized"
    fi

    echo
done

Best Practices

1. Tag All Resources: Enable cost allocation tracking
2. Set Budgets: Alert at 80% and 100% of budget
3. Monthly Reviews: Analyze costs every month
4. Right-Size Regularly: Check utilization quarterly
5. Use Savings Plans: For predictable workloads
6. Automate Cleanup: Delete unused resources
7. Enable Cost Anomaly Detection: Catch spikes early
8. Optimize Storage: Lifecycle policies for all S3 buckets
9. Monitor Trends: Track cost per user/transaction
10. Document Decisions: Why resources are provisioned

Requirements

# AWS CLI
pip install awscli boto3

# Configure
aws configure

# Set permissions (IAM policy):
# - ce:GetCostAndUsage
# - ec2:Describe*
# - cloudwatch:GetMetricStatistics
# - s3:List*

Cost Optimization Checklist

- [ ] EC2 instances right-sized
- [ ] Reserved Instances purchased for steady workloads
- [ ] S3 lifecycle policies configured
- [ ] Unattached EBS volumes deleted
- [ ] Unused Elastic IPs released
- [ ] Old snapshots cleaned up
- [ ] RDS instances right-sized
- [ ] Lambda memory optimized
- [ ] CloudWatch log retention set (7-30 days)
- [ ] Cost allocation tags applied
- [ ] Budget alerts configured
- [ ] Cost anomaly detection enabled

Metrics to Track

• Monthly cost: Trending down or flat
• Cost per user: Improving efficiency
• Unused resources: Minimal (<5%)
• RI/SP coverage: > 60% for predictable workloads
• Storage costs: Optimized with lifecycle policies
• Compute utilization: 40-70% average