MCP Servers Die Under Real AI Traffic

Wasted three days trying to get STDIO working in production. Save yourself the time.

STDIO Transport: Doesn't work. Send a bunch of requests, most fail. Get timeouts, connection refused errors, response times over 20 seconds. Not performance issues - actual failures.

Documentation doesn't mention this. Deploy to production and find out when AI agents start hitting your endpoints.

SSE Transport: Works better than STDIO but deprecated. Building on dead tech isn't smart.

Streamable HTTP: Only transport that works. Session strategy matters - get 30 RPS or couple hundred depending on how you configure it.

Tested on staging: shared sessions performed well, unique sessions were terrible. Huge difference.

Session pooling became required after the crashes. Not optional anymore.

AI Traffic Patterns Break Everything

AI agents hit differently than regular users. Burst traffic that overwhelms databases running default configs.

Single AI conversation creates dozens of parallel requests. Database was still on PostgreSQL defaults - around 100 max connections. AI agents tried opening way more. Got FATAL: sorry, too many clients already and everything crashed.

AI agents retry without backoff. Keep hammering until you add circuit breakers.

Connection Pool Reality

Static pools don't handle AI burst patterns. Found out when AI agent requested "analyze all customer data" and server tried opening more database connections than possible.

PostgreSQL defaults to around 100 connections - way too low for AI traffic bursts.

Memory Management Under AI Load

AI responses get massive. Started with small responses, then suddenly getting 30-50MB JSON payloads that crashed Node trying to serialize.

Production went down for hours when AI query returned huge dataset. Server died processing it all. Was customer data or product catalog, can't remember which.

Stream large responses or server dies when AI requests "all available data."

Session management determines if your server crawls or performs well. Tested both approaches: shared sessions much faster, unique sessions terrible performance. Huge difference.

STDIO: Don't Waste Time

Spent three days trying to get STDIO working in production. Don't repeat this mistake.

STDIO failures:

• Most requests fail outright
• Response times over 20 seconds
• Constant timeouts and connection errors

STDIO requires direct container attachment per connection. Each connection consumes dedicated resources. AI burst traffic overwhelms STDIO immediately.

Documentation doesn't warn about this. Find out in production when everything breaks.

SSE: Don't Build On Dead Tech

SSE worked better than STDIO - actually sustained some traffic. But it's deprecated tech.

If you're building new systems, skip SSE entirely. It's a dead end.

HTTP: The Only Option That Works

HTTP works, but session strategy determines if you crawl or actually perform well.

Shared Sessions: Fast when everything's working, handles tons of concurrent connections.

Unique Sessions: Terrible performance, maxes out quickly.

Choose wrong and server performance suffers badly.

Session Management That Actually Works

Session pooling prevents connection overhead. Pool size depends on your backend - database connections, API rate limits, memory constraints.

Started with 10 sessions. Increased when things started failing.

Dynamic scaling: Monitor pool utilization. Scale up when you're hitting limits, scale down when you're wasting resources. Don't overthink it.

Session affinity: AI conversations need consistent sessions. Route conversation turns to the same session to maintain context. Clean up old conversations to prevent memory leaks.

Circuit breakers: When backend gets overwhelmed, fail fast. Don't queue more requests. Set to trip after 5 failures in 30 seconds - whatever threshold keeps server alive.

Memory Problems with AI Responses

AI queries return massive datasets. One query returned huge dataset. Server died processing it all. Node ran out of heap space during serialization.

Stream large responses instead of buffering in memory. Send data in chunks. Set response limits at 10MB - anything bigger kills Node garbage collection.

Garbage collection matters more with AI workloads. Large objects stick around longer than expected. Force GC between large operations if necessary.

Kubernetes Reality

Container resource limits become real problems when AI agents start hammering your API.

• 512MB memory limits aren't enough for large JSON responses - Node 18 kept running out of memory
• CPU limits affect JSON parsing performance more than you'd think
• Network policies add latency that compounds under burst traffic

Service mesh adds a few milliseconds per request - not much for humans, but death by a thousand cuts for AI burst traffic. Consider bypassing the mesh for internal MCP communication.

Session optimization isn't about code elegance - it's about building infrastructure that doesn't fall over when AI agents start hammering your servers.

AI agents request huge datasets. One query returned massive dataset - crashed server. Node died trying to serialize enormous objects.

Signs Your Server Is About to Die

• Garbage collection pauses getting longer
• Response serialization taking forever
• Heap usage climbing above normal levels
• Event loop lag
• Memory not returning to baseline between requests

When you see these, your server is dying. Fix it before it crashes.

Stream Large Responses or Die

Don't serialize 30-50MB of JSON at once. Stream the response in chunks:

• Process data in batches (whatever batch size works for your data)
• Set hard response size limits - cap at 10MB because anything bigger breaks things
• Force garbage collection between chunks if needed
• Check memory pressure and back off when necessary

Stream data instead of buffering everything in memory. Difference between working and crashing.

Connection Pool Reality

Static pools don't work for AI burst traffic. AI agents hit your database with dozens of concurrent queries, then go silent for minutes. Static pools either waste resources or exhaust connections.

Use dynamic scaling:

• Start with base pool size (10 connections works)
• Scale up when things start breaking
• Scale down when wasting resources
• Set maximum limits based on database capacity (PostgreSQL starts complaining around 200-300 connections)

Monitor request patterns and adjust accordingly. Simple thresholds work better than complex algorithms.

Caching Strategy

Traditional caching fails with AI queries. AI agents ask the same question dozens of different ways:

• "Show customer data"
• "Display customer information"
• "Get customer records"
• "Fetch customer details"

These are semantically identical but miss in simple key-value caches.

Semantic caching can help but adds complexity. Simple caching with longer TTLs might be good enough.

Circuit Breakers

AI traffic can overwhelm backend services instantly. One AI agent requesting "comprehensive analysis" hammered APIs with hundreds of calls and killed external services.

Basic circuit breaker pattern:

• Trip when failing consistently (5 failures in 10 seconds works)
• Stay open for 30 seconds
• Test with one request (half-open)
• Close if test succeeds

Add burst detection - if you get tons of requests quickly, apply backpressure.

Monitoring That Actually Matters

Standard web metrics are useless for AI traffic - learned this when graphs looked fine while server was dying. Track these instead:

• Session pool utilization
• Response payload sizes
• Burst request rates
• Memory pressure during large responses
• Error rates during traffic spikes

Alert when session pool is getting hammered or memory starts climbing. These indicate server about to fall over.

The Bottom Line

AI workloads break traditional web optimization patterns. Burst traffic, large responses, and unpredictable query patterns require different approaches.

Web optimization techniques don't work for AI traffic. Learned this after server crashed three times in one week.

Hit a wall around 200 AI users. Teams that understand AI traffic patterns scale to thousands with same hardware.