Serverless Containers in Production - What Actually Works vs Marketing Bullshit

We moved 12 microservices from EKS to serverless containers thinking it would simplify our lives. It fucking didn't. Three months and fifteen grand in unexpected bills later, here's what actually happened.

AWS Fargate - The "Easy" Choice That Wasn't

Fargate promised to eliminate our Kubernetes nightmares. It just created new ones. No more node pool management, no more kubectl get nodes showing "NotReady" during the middle of the night. But Fargate's networking made us question our career choices.

The NAT Gateway Bill Shock

Got hammered with NAT Gateway charges - over $300 for what should've been a few GB of data transfer. Took us 3 days to figure out it was happening during container startup when they pull from ECR. Every single container boot was routing through the NAT Gateway instead of staying internal. VPC endpoints fixed it, but fuck if I know why that isn't the default.

Cold Starts Are Still Cold

Despite AWS marketing, our Spring Boot services took 11.3 seconds average to cold start on Fargate. 11 seconds of angry users hitting refresh. This Stack Overflow discussion captures the frustration perfectly - you're paying more than Lambda for slower cold starts.

Google Cloud Run - Great Until It Wasn't

Cloud Run looked amazing in demos. Deploy with one command, scale to zero, pay per request. Google's pricing model seemed reasonable until we hit production traffic.

The Concurrency Trap

Cloud Run defaults to 80 concurrent requests per instance. Sounds reasonable until you realize each request opens a new DB connection. Our PostgreSQL connection pool maxed out at 100 connections, so with 2 instances we were fucked. Lowering concurrency to 10 meant we needed 8x more instances during traffic spikes. Our bill went from $67 to $342 in one weekend. This Stack Overflow thread shows the same problem - Cloud Run's instance scaling math makes no sense for real apps.

Update - Multi-Container Support

As of May 2023, Cloud Run now supports sidecar containers. You can finally run database proxies, logging agents, and Nginx reverse proxies alongside your main app. But here's the catch - it adds complexity. Now you need to understand container dependencies, shared volumes, and network coordination. The simple "one container = one service" mental model is gone.

Azure Container Apps - Microsoft's Expensive Experiment

Azure Container Apps launched in 2022 as Microsoft's "we can do serverless containers too" answer to Fargate and Cloud Run. The pricing is brutal - $62.47/month per vCPU at full utilization vs Fargate's $29.50. That's a 111% Microsoft tax for the privilege of running on Azure.

The Windows Tax

The only reason to choose Azure Container Apps is Windows containers. If you're stuck running .NET Framework (not .NET Core), this is your only serverless option. Multiple Azure cost comparison discussions consistently show Azure is significantly more expensive - most teams avoid it for cost reasons.

Free Tier Gotcha

Azure promises 180,000 vCPU-seconds free per month. Sounds generous until you do the math - that's 50 hours total. Our dev environment container running 24/7 burned through the entire free tier in 2.08 days. Got a $47 surprise bill on day 3. This Stack Overflow thread shows we're not the only ones who fell for this.

What Actually Works in Production

After burning through fifteen grand learning these platforms, here's what we'd do differently:

Fargate is good for: Steady-state workloads where you can predict costs. Use VPC endpoints and ARM instances to cut costs.

Cloud Run is good for: Spiky HTTP APIs with proper connection pooling. Use minimum instances if you need consistent latency.

Azure Container Apps is good for: Windows containers and nothing else. The Microsoft tax is real.

Skip them all for: High-throughput workloads. Regular EKS/GKE/AKS clusters are still cheaper at scale.

AWS Fargate - Good Until The Networking Breaks

Fargate integrates well with the AWS ecosystem if you're already knee-deep in AWS services. The pricing is predictable - $0.04048/vCPU/hour means you can actually forecast costs without pulling your hair out.

But the networking will make you question your life choices. Want private containers? Cool, here's a $45/month NAT Gateway bill. Need custom VPC setups? Hope you enjoy reading 30-page networking guides.

We spent 2 weeks debugging why containers couldn't talk to RDS. The error was just "connection refused" - no details, no hints. Turns out the default security groups were blocking port 5432 even within the same VPC. AWS support's helpful response? "Have you reviewed the VPC networking best practices documentation?" 47 pages of networking docs to fix a checkbox. Real helpful, thanks.

Use Fargate if you're already on AWS, need predictable costs, and have a networking expert on your team who doesn't mind pain.

Google Cloud Run - Amazing Until You Scale

Cloud Run deployment is stupid simple: gcloud run deploy. Auto-scaling actually works, and scaling to zero saves money when nobody's hitting your API. The pricing model makes sense for spiky workloads.

Single container per service kills most real applications UPDATE: As of May 2023, Cloud Run supports sidecars. But honestly? The multi-container setup is a pain in the ass. You need to configure container dependencies, manage shared volumes, and debug networking between containers. What used to be simple became complicated.

Our Node.js API worked great in testing - then Black Friday traffic hit over 1,200 concurrent users and everything went to shit. Cloud Run was spinning up 30+ instances, each opening 5-10 DB connections. PostgreSQL hit its 200 connection limit and started refusing new connections with FATAL: sorry, too many clients already. We had to rewrite our entire connection pooling from scratch using a shared Redis connection pool. Spent 14 hours that weekend fixing what should've been anticipated.

Use Cloud Run if you have simple HTTP APIs, don't mind debugging YAML container dependencies, and your traffic patterns are unpredictable.

Azure Container Apps - Expensive But It Runs Windows

Azure Container Apps is the only serverless platform that runs Windows containers. Stuck with .NET Framework 4.8? This is your only option. KEDA autoscaling is actually pretty clever.

But the pricing is brutal. At $0.0571/vCPU-hour (about $42/month per vCPU plus memory costs), you're looking at 40-50% higher costs than Fargate. Microsoft's reasoning: "It's enterprise grade!" Translation: "We know you're locked in, so pay up."

We deployed a simple .NET Core API (2 vCPU, 4GB RAM) and got a bill for almost $200 monthly for what would cost $64.50 on Fargate. The free tier lasted exactly 2.8 days before billing kicked in with zero warning. Azure's cost alerting sent us an email 6 hours after we'd already blown through $50 in overages. You find out about expensive mistakes after your credit card gets charged.

Use Azure Container Apps if you need Windows containers, you're already paying Microsoft everything anyway, or your compliance requirements force you into Azure.

Other Options That Don't Suck

Civo spins up clusters in 90 seconds and doesn't try to nickel-and-dime you to death. Great for development and testing. Production use is limited by their small geographic footprint, but if you're in the UK, it's solid.

WebAssembly promises sub-millisecond cold starts and 10x memory efficiency. Fermyon's Spin and Wasmtime are leading the charge.

WASM is cool in demos but the ecosystem is still tiny. Need to connect to PostgreSQL? Good luck finding a mature WASM driver. Want to use your favorite logging library? Probably doesn't exist yet.

Stick with the big three for production workloads. Try the emerging platforms for side projects and bleeding-edge experiments.

The Real Talk Recommendations

After burning money on all three platforms and debugging production outages, here's what actually works:

For Steady Workloads: AWS Fargate Wins
Fargate costs $29.55/month per vCPU running 24/7. That's exactly half of Azure's $62.47 and way more predictable than Cloud Run's "request-based pricing" that turns into instance-based billing when you hit real traffic.

But those hidden costs will fuck you: NAT Gateway bills ($45.50/month), Application Load Balancer fees ($18.50/month), CloudWatch logging ($0.50/GB that adds up fast), and data transfer charges nobody mentions. Sony cut their costs in half switching to Fargate, but they also have 47 engineers who've memorized AWS networking docs.

For Spiky Traffic: Google Cloud Run
Cloud Run's pay-per-request model is perfect when your traffic is all over the place. Scales to zero when nobody's hitting your API, scales up when you get slashdotted. Cold starts averaged 2.7 seconds for our Node.js apps - fast enough for APIs, too slow for user-facing pages.

The deploy experience is actually great - gcloud run deploy and you're live in 45 seconds with HTTPS included. But multi-container support turned simple deployments into debugging YAML container dependencies at 2am when your sidecars won't start.

For Windows Containers: Azure Container Apps (Your Only Choice)
Azure costs $0.0571/vCPU-hour ($41.71/month per vCPU plus $0.000003/GB-second for memory) - 41% more expensive than Fargate for the same resources. But if you're stuck running .NET Framework 4.8 on Windows containers, this is literally your only serverless option.

The KEDA autoscaling is actually pretty clever - you can scale on custom metrics like queue length. Too bad about the Microsoft tax making everything expensive as hell.

The New Kids on the Block

Civo - Fast and Cheap (For Now)
Civo spins up clusters in 90 seconds and doesn't nickel-and-dime you to death. Great for development work. Some companies report massive cost savings, but they're mostly UK-based with limited global reach.

Perfect for testing and development environments. Production? Depends how much you trust a smaller provider with your uptime.

WebAssembly - Cool Demos, Tiny Ecosystem
WebAssembly platforms promise sub-millisecond cold starts and stupid-efficient memory usage. The benchmarks look amazing for AI workloads.

Reality check: The ecosystem is still tiny. Need to connect to PostgreSQL? Good luck finding a mature WASM driver that doesn't suck. Want to use your favorite logging library? Probably doesn't exist yet.

How to Actually Choose

Forget the consultant recommendations. Here's how to actually pick:

Traffic patterns matter most - spiky shit needs Cloud Run, steady workloads go with Fargate, and if you need Windows containers you're stuck with Azure.

If you're already deep in one ecosystem, stay there. All-in on AWS? Use Fargate. Google everything? Cloud Run. Microsoft shop? Azure tax it is.

Team skills matter more than features. Kubernetes veterans should stick with regular GKE/EKS. Docker-only teams love Cloud Run's simplicity. Traditional ops teams handle Fargate better.

Compliance requirements will override everything else. Government work means AWS GovCloud. European data laws need EU regions. Microsoft compliance requirements lock you into Azure whether you like it or not.

What's Coming Next (Probably)

WebAssembly Will Get Real: Right now it's all demos and benchmarks. By 2026, expect AWS and Google to launch proper WASM serverless platforms. The performance gains are too good to ignore once the ecosystem catches up.

Bills Will Force Better Pricing: When serverless hits 20-30% of your cloud budget, CFOs start asking questions. Platforms will add more transparent pricing and better cost controls because they have to.

Edge Computing Everywhere: 5G means your app needs to run everywhere. Cloudflare Workers and Fastly Compute are already showing how this works.

Bottom line: Pick a platform that's not going extinct. AWS, Google, and Microsoft aren't going anywhere. The startups? Civo might survive, but most will get acqui-hired for their talent while their platforms get shut down. Don't bet your production workloads on companies with Series B funding.