Every few months someone in a Slack channel posts a screenshot of their AWS bill and the thread goes quiet. Not the good kind of quiet — the kind where everyone is doing mental arithmetic on their own infrastructure spend.
The promise was simple: pay for what you use, stop buying hardware, let someone else worry about the data center. For a lot of teams in 2010, that was genuinely liberating. But it's 2024, and the question is cloud computing still cost effective deserves a more honest answer than the one your cloud vendor's ROI calculator will give you.
This isn't a hit piece on AWS, Azure, or GCP. It's a pattern I keep seeing: companies that moved to the cloud for cost reasons are now quietly moving workloads back — or at least asking hard questions. Let me walk through where the math works, where it doesn't, and what I'd actually do.
The Original Deal Was Real — For a Specific Kind of Company
Small startups, variable-traffic apps, teams without a dedicated ops person — for all of these, cloud economics made obvious sense in the early days. You weren't buying a $40,000 server for a product that might not exist in 18 months. You were renting compute by the hour. The flexibility premium was worth paying.
That logic still holds for genuinely unpredictable workloads. If your traffic can spike 10x during a product launch and flatline afterward, on-premises hardware is a terrible fit. You'd either over-provision and waste money or under-provision and fall over at the worst moment.
But here's the thing most cloud vendors don't advertise: the flexibility premium compounds. EC2 On-Demand pricing for an m6i.2xlarge (8 vCPU, 32 GB RAM) runs about $0.384/hour as of mid-2024. That's roughly $280/month, or $3,360/year — for one instance. A comparable bare-metal server from a mid-tier hosting provider costs around $150–180/month. The gap widens when you add managed databases, load balancers, NAT gateways, and the data egress fees that nobody budgets for until the first bill.
Where Cloud Costs Actually Come From
The line item that surprises people most isn't compute. It's everything around compute.
Data egress. AWS charges $0.09/GB for data transferred out to the internet (as of 2024, after the first 100 GB free tier). If you're running a media-heavy app or doing large data exports, this adds up fast. A company moving 50 TB/month pays about $4,500 just to get their own data out. GCP and Azure are similar. This is, frankly, a captivity tax disguised as an infrastructure cost.
Managed services markup. Amazon RDS for a db.m6g.2xlarge with Multi-AZ costs around $0.96/hour — nearly $700/month for one database instance. A self-managed PostgreSQL 16 installation on a dedicated VM with the same specs costs a fraction of that. You're paying for convenience and the ops burden you're offloading, which is sometimes worth it and sometimes isn't.
Idle resources. The dirty secret of cloud adoption is that most teams are terrible at turning things off. Dev environments that run 24/7. Staging clusters that mirror production. Forgotten load balancers from a project that ended eight months ago. One study by Flexera (their 2023 State of the Cloud report) found that organizations waste an average of 32% of their cloud spend. That's not a rounding error.
When the Answer to "Is Cloud Computing Still Cost Effective" Is Yes
I want to be fair here. There are scenarios where cloud is clearly the right economic choice.
Early-stage startups. If you're pre-product-market-fit, the last thing you need is capital tied up in hardware. AWS Free Tier plus a modest EC2 instance gets you to launch. The flexibility to scale (or shut down) without sunk costs is genuinely valuable.
Bursty, unpredictable workloads. Machine learning training jobs, batch processing pipelines, seasonal e-commerce traffic — these are natural fits. Spot Instances on AWS or Preemptible VMs on GCP can cut compute costs by 60–90% for fault-tolerant workloads. I've seen ML teams run training jobs at $0.30/hour on Spot that would cost $2.00/hour on-demand. That's real money.
Teams without infrastructure expertise. Managed Kubernetes (EKS, GKE, AKS), managed databases, managed caches — if your team has two engineers and neither wants to become a DBA, you're paying for peace of mind. That's a legitimate business decision.
Geographic distribution. Spinning up a replica in Singapore to serve Southeast Asian users would take months on-prem. On AWS it takes an afternoon. If your product requires global presence, cloud's infrastructure footprint is genuinely hard to replicate.
When the Math Turns Against You
The calculus shifts when workloads become predictable and large. This is the repatriation story you're starting to hear more often.
DHH and the 37signals team made headlines in 2023 when they published their cloud exit numbers: they estimated saving around $7 million over five years by moving Basecamp and HEY off AWS onto their own hardware. Their workload was large, stable, and well-understood. The flexibility premium was pure overhead.
You don't have to be 37signals for this to apply. If you're running the same 20 instances at roughly the same utilization every month for two years, you're not using the cloud's flexibility — you're just renting at retail prices. Reserved Instances and Savings Plans help (1-year RIs typically cut costs 30–40%), but you're still paying a premium over owning hardware outright over a 3–5 year horizon.
The break-even point varies, but a rough rule I use: if your monthly cloud bill exceeds $15,000–20,000 and your workload is stable, it's worth running the numbers on dedicated hosting or colocation. Not necessarily full repatriation — but at least a hybrid model.
A Practical Audit You Can Do This Week
Before making any infrastructure decisions, do this:
- Pull your last three months of cloud bills. Break them down by service, not just total.
- Tag everything. If you don't have resource tagging in place, you're flying blind. AWS Cost Explorer is useful here; so is the open-source tool Infracost if you're using Terraform.
- Identify your top five cost drivers. Usually it's compute, RDS, data transfer, and two surprises.
- Separate fixed from variable costs. Fixed costs that don't flex with traffic are candidates for Reserved Instances or migration.
- Calculate your effective hourly rate per vCPU and per GB of RAM. Compare it to a bare-metal quote from Hetzner, OVHcloud, or a colocation provider.
This exercise alone has saved teams I know thousands of dollars a month — not by leaving the cloud, but by right-sizing and eliminating zombie resources.
The Honest Answer
Is cloud computing still cost effective? It depends on your stage, workload shape, and how disciplined your team is — but that's not a cop-out answer, it's a diagnostic one.
For most startups under $5k/month in cloud spend: yes, absolutely. The flexibility and reduced ops burden justify the premium.
For companies with stable, large workloads paying $30k+/month: probably not purely on cost grounds. You should at least run the numbers.
For everyone in between: the cloud is cost effective if you treat it like a utility you actively manage, not a credit card you leave on autopilot.
The vendors have every incentive to make migration in feel frictionless and migration out feel painful. Data egress pricing is the most obvious example of that asymmetry. Go in with open eyes.
For more on how infrastructure decisions compound over time, see my earlier piece on the hidden costs of developer tooling.
What to Do Tomorrow
Open your cloud console, pull last month's bill, and spend 30 minutes categorizing it by service. If data transfer or idle compute shows up in your top three costs, you have an immediate optimization opportunity — no architecture changes required. That single hour of audit work is the most cost-effective thing you can do with your infrastructure this week.