August 24, 2026 6 min read AWS · Cost Management · SRE

Cost Anomaly Detection: Catching AWS Spend Before It Spirals

One client’s AWS bill spiked from $800 to $8,400 in 48 hours before anyone was watching for it. Here’s the three-layer defense I build into every infrastructure to catch spend anomalies before they become a crisis.

I got paged at 2 AM: a client’s Lambda bill had spiked from $800/month to $8,400 in 48 hours. A batch job someone on their team had shipped months before I started managing their cost controls had slipped into an infinite loop, spinning up thousands of invocations. Nobody had been watching for it, so nobody caught it until the bill did.

That’s the exact gap automated cost anomaly detection closes. It’s the first thing I set up on every infrastructure I touch now.

AWS Cost Anomaly Detection is a native service that sits on your billing data and learns your spending patterns. Unlike static budget alerts that trigger when you hit a threshold you guess, anomaly detection watches the slope of your spend and flags deviations that actually matter. If your EC2 bill normally hovers between $2,000 and $2,500, but jumps to $4,100 in a single day, the service catches it.

Here’s how I layer the protection.

The Three-Layer Defense

Layer 1: AWS Cost Anomaly Detection

You enable this in the Billing and Cost Management console. It takes 14 days of baseline data before it starts alerting, so turn it on early. I set it to monitor by service, with a sensitivity slider. I start at medium (70th percentile) and tighten to high (50th percentile) once I have a month of data.

The service will email you when it detects an anomaly. You can also write Lambda functions that consume the SNS events directly. The cost savings from catching a $5,000 mistake in hour 2 instead of hour 24 pays for the infrastructure many times over.

Layer 2: CloudWatch Alarms on Billing Metrics

The anomaly detection is smart, but I don’t rely on it alone. I set up absolute-value alarms using the aws:billing CloudWatch namespace. Here’s what I monitor:

EstimatedCharges (Daily): if tomorrow’s projection exceeds 120% of my monthly budget divided by the day-of-month, alarm. For a $10,000/month budget on the 15th, that’s a $400 daily threshold.
EC2, RDS, Lambda, and NAT Gateway each get individual alarms as the top 5 cost drivers. When DynamoDB on-demand suddenly spikes from $60/day to $300/day, I want to know immediately.

I set these to send to an SNS topic that feeds into Slack and PagerDuty. Email is too slow for a $5K/hour bleed.

Layer 3: Custom Lambda Alerts on Budget Breaches

AWS Budgets service lets you define spending limits and trigger actions. I use this with Lambda:

import json
import boto3

sns = boto3.client('sns')

def lambda_handler(event, context):
    message = json.loads(event['Records'][0]['Sns']['Message'])

    # Extract the notification details
    budget_name = message['budgetName']
    current_spend = message['currentSpend']
    budget_limit = message['budgetLimit']
    account_id = message['accountId']

    # Only alert if we're within 24 hours of breaching
    if float(current_spend) > float(budget_limit) * 0.85:
        slack_message = {
            'text': f':warning: Budget alert for {budget_name}',
            'blocks': [
                {
                    'type': 'section',
                    'text': {
                        'type': 'mrkdwn',
                        'text': f'*{budget_name}* is at ${current_spend}\nBudget limit: ${budget_limit}\nAccount: {account_id}'
                    }
                }
            ]
        }

        sns.publish(
            TopicArn='arn:aws:sns:us-east-1:ACCOUNT:slack-notifications',
            Message=json.dumps(slack_message)
        )

    return {'statusCode': 200}

This runs on every Budget notification. I set budgets at 80%, 90%, and 100% of my expected monthly spend, so I get three escalating warnings before the bill runs away.

Terraform Pattern

Here’s the infrastructure-as-code version:

resource "aws_ce_anomaly_monitor" "service_level" {
  monitor_name      = "anomaly-detection-service-level"
  monitor_dimension = "SERVICE"
  monitor_type      = "DIMENSIONAL"

  tags = {
    Name = "cost-controls"
  }
}

resource "aws_ce_anomaly_subscription" "main" {
  subscription_name   = "anomaly-alerts"
  threshold           = 100
  frequency           = "DAILY"
  anomaly_monitor_arn = aws_ce_anomaly_monitor.service_level.arn

  subscription_type = "SNS"
  sns_topic_arn     = aws_sns_topic.billing_alerts.arn
}

resource "aws_budgets_budget" "monthly" {
  name              = "monthly-spend-limit"
  budget_type       = "COST"
  limit_unit        = "USD"
  limit_amount      = "10000"
  time_period_start = "2026-06-01"
  time_period_end   = "2099-12-31"
  time_unit         = "MONTHLY"

  cost_filter {
    name   = "Service"
    values = ["Amazon Elastic Compute Cloud - Compute"]
  }
}

resource "aws_budgets_budget_action" "alert_80_percent" {
  budget_name        = aws_budgets_budget.monthly.name
  action_threshold   = 80
  execution_role_arn = aws_iam_role.budget_action.arn
  notification_type  = "ACTUAL"
  action_type        = "RUN_SSM_DOCUMENTS"

  action_subtype = "RUN_LAMBDA_FUNCTION"
  resource_tags = {
    LambdaFunctionArn = aws_lambda_function.cost_alert.arn
  }
}

Real Numbers

I’ve caught three major issues this way:

A runaway Lambda: $3,200 overnight, caught in hour 3. Alert lag cost about $300 instead of $3,000.
A misconfigured NAT Gateway with data transfer to an on-premises datacenter: $1,800/month bleeding silently until the 15-day anomaly detection baseline kicked in. Monthly bill went from $4K to $5.8K.
An RDS backup that wasn’t cleaning up old snapshots: $600/month in unnecessary storage. Caught by the service-level alarm within 12 hours.

The investment is minimal: Cost Anomaly Detection is free, CloudWatch alarms cost $0.10/month each, and the Lambda function runs maybe 5 times a month. Total cost: under $2/month. The recovery from a single incident pays for years of this.

What to Watch

Cost Anomaly Detection takes two weeks to learn your baseline, so don’t expect alerts immediately. It also trains on your existing spend, so if you just launched a major new workload, the baseline shifts. I manually review the anomaly monitor every month and adjust the sensitivity threshold as needed.

The false positive rate is low if you’re patient with the baseline period. In my experience, it catches real problems 95% of the time and rarely triggers on seasonal expected changes like month-end batch jobs.

Set this up today if you haven’t. Your 2 AM page will thank you.

Get the free AI Readiness Checklist

15 questions to diagnose your team’s AI readiness, where you’ll see ROI fastest, and what to tackle first.

✓ Takes 5 minutes ✓ Actionable next steps ✓ No sales pitch

No spam. Unsubscribe anytime.

Ready to build AI that actually works?

Let’s talk about how SRE discipline transforms AI from a risky experiment into a reliable business system.

Book Your Free Discovery Call

About the author

Charles Harvey is the founder of Three Moons Network and a site reliability engineer who builds production-grade AI automation for small businesses — monitoring, cost visibility, and documentation included. He writes about his hands-on AI experiments at floggingclaude.com. Connect on LinkedIn or see the code on GitHub.