Fly.io Performance Optimization - CPU Quotas, Throttling, and Scaling Guide

On January 14, 2025, Fly.io fully enabled CPU quota enforcement changes that have fundamentally altered how applications perform on their platform. What used to be 30-second deploys now take 8+ minutes for Django, Rails, and Node.js applications, catching developers completely off-guard with zero email notification about the breaking change.

Understanding the New CPU Throttling System

The core change is brutal in its simplicity: shared vCPUs are now limited to 1/16th of a CPU core (6.25% baseline), while performance vCPUs get 100% of a dedicated core. This means a shared-cpu-1x machine that previously had access to burst CPU power can now only sustain 62.5 milliseconds of CPU time per second.

The throttling system works on 80ms cycles, where each cycle grants you 5ms of CPU time on a shared-cpu-1x. Any time you don't use gets banked as "burst balance" - but with a low accrual rate that makes startup-heavy applications suffer dramatically.

The Real-World Impact

Deploy Time Explosions: Applications that previously deployed in 30 seconds now take 8-20 minutes to boot because startup processes get throttled immediately. Django apps loading models, Rails apps precompiling assets, and Node.js apps warming up V8 all hit this wall hard.

Production Outages: Multiple users reported immediate production outages when the enforcement went live. Rolling deployments failed because new instances couldn't pass health checks within reasonable timeframes.

Scaling Confusion: A shared-cpu-8x machine still gets the same 6.25% baseline as shared-cpu-1x, making the scaling meaningless for CPU-bound workloads. Users found themselves forced to upgrade to performance instances just to deploy successfully.

Why Fly.io Made This Change

The Predictable Processor Performance initiative aimed to address "noisy neighbor" problems where some applications consumed excessive CPU, affecting others on the same hardware. The change aligns with industry standards - AWS Lambda has similar throttling, Google Cloud Run limits concurrent requests, and most cloud providers limit shared resources.

However, Fly.io completely botched this rollout. Despite their bullshit claims that "a tiny fraction of organizations" would be affected, the change broke apps everywhere. My production Rails app shit the bed middle of the day with zero warning - I found out when users started complaining about 10-second page loads, not from any email from Fly.io.

Zero fucking email notifications about a change this massive violates every change management practice that real platform providers follow.

The Burst Balance System

Fly.io provides a "burst balance" mechanism to soften the throttling impact. Unused CPU time accumulates and can be spent in bursts, but the math is unforgiving:

• Idle Accumulation: A completely idle shared-cpu-1x accrues only 3.75 minutes of burst balance per hour, similar to AWS EC2 burst credits but with much lower accrual rates
• Startup Penalty: New machines start with just 5 seconds of burst balance, insufficient for most application startup sequences and unlike Google Cloud Run's generous startup allowances
• Reset on Restart: Restarting a machine after startup mysteriously works better - took me 3 hours of debugging to figure this shit out, but restart the machine and suddenly it's fast again

Performance vs. Shared CPU Economics

So now you're stuck choosing between broken or expensive:

• shared-cpu-1x: $2/month but doesn't work
• performance-1x: $7/month but actually functions

shared-cpu-1x costs $2/month but doesn't work, performance-1x costs $7/month but actually functions. Do the math - about 4x more expensive for basic functionality that worked fine before their throttling disaster. AWS and Google do similar throttling, but they don't spring it on you without warning like Fly.io did.

The CPU quota disaster broke every optimization strategy that actually worked before January 2025. I spent 6 hours trying every Docker trick in the book and still got 6-minute deploy times for a basic Node.js app. Here's what actually works now that Fly.io throttled shared CPUs into oblivion.

Docker Tricks That Actually Help Now

I learned this the hard way - put all your heavy shit in the Dockerfile, don't run it at startup where Fly.io will throttle you to death. Pre-compile everything during the build phase.

## Good: Pre-build assets during image creation
RUN npm run build:production
RUN bundle exec rails assets:precompile

## Bad: These run every time the container starts
CMD [\"sh\", \"-c\", \"npm run build && rails s\"]

Lazy loading saved my ass - don't load everything at startup. Our Django app was loading dozens of models at boot and it would timeout. Now I lazy-load models only when needed and startup is actually reasonable.

Background Processing: Move CPU-intensive startup tasks to background jobs that can run after the health checks pass. I use Sidekiq for Ruby apps, Celery for Python, and Bull for Node.js to offload the heavy shit from startup.

Memory Tricks That Actually Help

Garbage Collection Tuning: Language-specific garbage collection can trigger CPU spikes that push you over quota limits. Configure GC settings for your runtime:

• Node.js: Use `--max-old-space-size` to control V8 heap limits and reduce GC pressure
  I spent a weekend tuning Ruby GC settings and MALLOC_ARENA_MAX=2 made the biggest difference - learned that the hard way after Rails kept timing out on startup. For Python, disabling GC during Django startup actually helps with the throttling.

Memory Profiling: High memory usage triggers swapping, which increases CPU consumption and makes throttling worse. Monitor these Fly.io metrics:

• `fly_instance_memory_swap_free` - Should stay close to total swap
• `fly_instance_memory_mem_available` - Should remain above 20% of total RAM
• `fly_instance_memory_cached` - Higher is better for file-heavy applications

Use tools like Valgrind for C/C++, memory_profiler for Python, or Node.js built-in heap profiling to identify memory bottlenecks that contribute to CPU throttling.

How to Size Your Instances Without Going Broke

The new economics make instance sizing critical. Here's what I figured out after burning through my budget testing this shit:

• Hobby projects: shared-cpu-1x with autostop enabled, accept slow deploys
• Development/staging: shared-cpu-2x or shared-cpu-4x for slightly better startup, but consider blue-green deploys
• Production: performance-1x minimum for reliable deployment and operation
• High-traffic production: performance-2x or higher based on actual load testing

Regional Distribution: Spread instances across regions to reduce individual instance load. Fly.io's WireGuard mesh networking makes this operationally simple, and regional distribution can improve both performance and reliability.

Autoscaling Configuration: Set up autoscaling based on CPU metrics rather than just concurrent connections. Monitor fly_instance_cpu metrics and scale before hitting throttling limits.

Deployment Strategy Optimization

Blue-Green Deployments: If you don't use volumes, blue-green deployment strategy works great until you realize you lose persistent storage - which rules out most real applications. I tried this for our Rails app and it worked beautifully until we needed file uploads and realized we'd have to rebuild our entire storage architecture. Also breaks session storage if you're not using Redis or some external store.

[build]
  strategy = \"bluegreen\"

[http_service]
  processes = [\"web\"]
  internal_port = 8080
  force_https = true

  # Give instances more time to warm up
  [http_service.http_options]
    grace_period = \"30s\"

Rolling Deployment Tuning: If you must use rolling deployments (volumes, stateful applications), adjust these settings:

• Increase health check timeouts to accommodate slower startups
• Reduce the number of concurrent replacements during deployment
• Consider using fly deploy --strategy immediate for faster rollbacks

Monitoring and Alerting Setup

The metrics that actually matter (learned this the expensive way):

• fly_instance_cpu_throttle - Throttling time indicates performance issues
• fly_instance_cpu_balance - Low balance predicts future throttling
• fly_app_http_response_time_seconds - End-user impact measurement
• fly_edge_http_response_time_seconds - Global performance perspective

Set up alerts for:

• CPU throttling over 10% of total time
• Burst balance below 60 seconds
• Response times above your SLA thresholds
• Deploy times exceeding expected duration.

Framework-specific bullshit I had to figure out (after 2 hours of prod downtime):

Django Applications:

• Use --lazy-apps flag with Gunicorn to delay model loading
• Pre-build static files in Docker image, not at startup
• Consider django-extensions for profiling startup bottlenecks
• Tune DATABASES['default']['CONN_MAX_AGE'] to reduce connection overhead

Rails Applications:

• Enable config.eager_load = true in production to front-load initialization
• Use bootsnap gem for faster boot times
• Pre-compile assets in Docker build, not at startup
• Consider derailed_benchmarks gem for memory profiling

Node.js Applications:

• Use --expose-gc flag and manually trigger garbage collection during idle periods
• Implement clustering with PM2 or Node.js cluster module to distribute load
• Pre-build and cache expensive computations in the Docker image
• Monitor V8 heap usage with --heap-prof during development.

Cost Optimization Strategies

Hybrid Instance Strategy: Use performance instances for critical services and shared instances for less demanding workloads like background workers or admin interfaces.

Autostop Configuration: For development and staging environments, autostop functionality can significantly reduce costs, despite the cold start penalty.

Resource Right-sizing: Regularly review your instance utilization. Over-provisioned performance instances are expensive, but under-provisioned shared instances are unreliable. Use the metrics data to find the sweet spot.

The new CPU quota reality has made performance optimization from a nice-to-have into a mission-critical requirement. Applications that worked fine with previous Fly.io defaults now require careful tuning and potentially higher-tier instances to maintain acceptable performance and deployment reliability.