API Rate Limiting: Production Implementation Guide

Critical Implementation Requirements

Algorithm Selection Matrix

Algorithm	Memory Impact	Burst Handling	Accuracy	Implementation Complexity
Fixed Window	Minimal RAM usage	Vulnerable to burst attacks at window boundaries	Weather forecast accuracy	Copy-paste simple
Sliding Window	RAM intensive	Excellent burst protection	High precision	Complex debugging required
Token Bucket	Light memory footprint	Perfect for bursty traffic	Production adequate	Moderate complexity
Leaky Bucket	Memory intensive	Complete traffic smoothing	Obsessive precision	Maximum complexity

Production-Critical Failure Modes

Memory Exhaustion: Sliding window algorithms store every request timestamp - explodes RAM with high traffic

• Solution: Set Redis maxmemory with allkeys-lru policy
• Warning: Without memory limits, rate limiter will consume all available RAM

Redis Connection Leaks:

• Node.js 18.12.0: Memory leak with Redis connections - crashes every 8 hours
• go-redis v8: Connection leaks cause 50MB to 2GB memory growth over 48 hours
• Python 3.8 redis.asyncio: Random connection drops cause "connection pool exhausted"

Clock Skew in Kubernetes: Different pod times break sliding window algorithms

• Impact: Rate limiter blocks legitimate traffic or fails completely
• Solution: Use NTP sync or switch to token bucket algorithm

Implementation Specifications

Node.js Production Configuration

Required Dependencies:

npm install express ioredis
# CRITICAL: Don't use express-rate-limit for distributed systems
# CRITICAL: Don't use old 'redis' package - has connection leaks

Version Requirements:

• Node.js: 18.15.0+ (18.12.0 has memory leak)
• ioredis: Latest (old redis package leaks connections)

Production Implementation Checklist:

• ✅ Fail open when Redis unavailable (better extra traffic than blocked users)
• ✅ Pipeline Redis operations for atomicity
• ✅ Set key expiration to prevent memory leaks
• ✅ Handle proxy IP extraction (X-Forwarded-For, X-Real-IP)
• ✅ Exclude health checks from rate limiting
• ✅ Include Retry-After header in 429 responses

Redis Configuration for Production

Memory Protection:

command: redis-server --appendonly yes --maxmemory 100mb --maxmemory-policy allkeys-lru

Connection Pool Settings:

{
  retryDelayOnFailover: 100,
  enableReadyCheck: false,
  maxRetriesPerRequest: 1,  // Don't retry forever
  connectTimeout: 5000,
  commandTimeout: 3000      // Fail fast on slow Redis
}

Algorithm Implementation Patterns

Token Bucket (Recommended for Production):

• Capacity: 10 tokens
• Refill rate: 1 token per window
• Window: 60 seconds
• Memory efficient with burst handling

Fixed Window (Simplest):

• Key format: rate_limit:{client_id}:{timestamp_minute}
• Atomic increment with expiration
• Vulnerable to boundary burst attacks

Sliding Window (High Traffic):

• Uses Redis sorted sets
• Stores timestamp for each request
• High memory usage but precise control

Deployment Specifications

Docker Configuration

Dockerfile Requirements:

• Use Node.js 18.15-alpine (18.12 memory leaks)
• Wait for Redis availability before starting
• Run as non-root user for security

Docker Compose Setup:

redis:
  image: redis:7-alpine
  command: redis-server --appendonly yes --maxmemory 100mb --maxmemory-policy allkeys-lru
  volumes:
    - redis_data:/data

Monitoring and Alerting

Critical Metrics:

• Requests allowed vs blocked ratio
• Redis connection errors
• Block rate percentage
• Response time impact

Alert Thresholds:

• Block rate >20%: Possible attack or limits too strict
• Redis errors >0: Rate limiting degraded
• Memory usage >80%: Scale Redis or optimize algorithm

Common Failure Scenarios and Solutions

Redis Failures

Symptom: "Redis HMGET failed" errors
Cause: Redis unavailable or slow
Solution: Fail open, log errors, implement circuit breaker

Symptom: Connection timeout errors
Root Causes:

1. Using deprecated redis package instead of ioredis
2. Node.js 18.12.0 memory leak
3. Redis maxclients set too low

Traffic Pattern Issues

Symptom: Rate limiter blocks everyone with low traffic
Cause: Clock skew between servers
Solution: Use NTP sync or token bucket algorithm

Symptom: Works in development, fails in production
Cause: Load balancer modifying IP headers
Solution: Test actual X-Forwarded-For values in production

Performance Degradation

Symptom: Sliding window algorithm becomes slow
Cause: Too many Redis operations per request
Solution: Switch to fixed window or implement client-side caching

Symptom: Memory explosion in Redis
Cause: No expiration on rate limit keys
Solution: Set TTL on all keys, implement memory limits

Security Considerations

IP-Based Limiting Challenges

NAT/Proxy Issues: Multiple users share single IP

• Solution: Higher limits for shared IPs, API key-based limiting for authenticated users
• Detection: Monitor for unusually high traffic from single IPs

Load Balancer Header Manipulation:

• Risk: LB strips or modifies X-Forwarded-For
• Mitigation: Test header values with real traffic, implement fallback IP detection

Fail-Safe Implementation

Redis Unavailable: Always fail open
Performance Degradation: Implement circuit breaker
Attack Detection: Alert on >50% block rate from specific IPs

Debugging Production Issues

Redis Diagnostics

redis-cli ping                    # Check connectivity
redis-cli --latency              # Check Redis performance
redis-cli memory usage           # Check memory consumption
redis-cli ttl your_key          # Verify key expiration

Application Diagnostics

• Monitor actual IP values received vs expected
• Log rate limit decisions for analysis
• Track Redis operation timing
• Verify header extraction in production environment

Kubernetes-Specific Issues

• Check pod system time synchronization
• Verify service mesh header handling
• Monitor network policy impacts
• Test during rolling deployments

Resource Requirements

Performance Specifications

• Fixed Window: ~1ms per request, minimal RAM
• Token Bucket: ~2ms per request, low RAM usage
• Sliding Window: ~5ms per request, high RAM usage
• Redis Memory: 1MB per 10,000 active rate limit keys

Infrastructure Costs

• Redis Instance: 100MB RAM minimum for production
• Network Overhead: ~1KB per rate-limited request
• Monitoring: Additional 10% CPU for metrics collection

Scalability Thresholds

• Single Redis: Up to 100,000 requests/second
• Redis Cluster: Required above 100,000 RPS
• Memory Planning: 1GB Redis per 1M requests/hour for sliding window

Critical Success Factors

1. Fail Open Strategy: Never block all traffic due to rate limiter issues
2. Comprehensive Monitoring: Track both technical metrics and business impact
3. Gradual Rollout: Start with loose limits, tighten based on traffic patterns
4. Health Check Exclusions: Never rate limit monitoring endpoints
5. Production Testing: Load test rate limiter before deployment