Django Docker Compose Production Deployment - AI-Optimized Guide

Critical Failure Modes and Solutions

Container Exit Scenarios

• Immediate container exit: Missing environment variables cause KeyError or django.core.exceptions.ImproperlyConfigured
• SECRET_KEY errors: Container starts, Django reads settings, hits missing SECRET_KEY, exits with code 1
• Windows line endings: .env.prod file with \r\n breaks variable parsing - fix with dos2unix .env.prod
• Permission issues: Container cannot read env file - check file permissions

Database Connection Failures

• "relation does not exist": Django queries non-existent tables - run migrations after container startup
• Service communication: Use service names (db) not localhost/127.0.0.1 in connection strings
• Connection timeouts: Add health checks to depends_on configuration
• Version incompatibility: PostgreSQL version changes require data volume cleanup or migration

Static File Issues

• 404 errors on CSS/JS: Forgot to run collectstatic after deployment
• Shared volume mounting: Ensure static_volume mounted in both web and nginx services
• Cache problems: Aggressive caching headers (expires 1y) serve old files after changes

Production Architecture Requirements

Essential Components

• Gunicorn: Replaces Django's single-threaded runserver with multi-process workers
• PostgreSQL: MVCC prevents write-blocking, supports connection pooling and advanced indexing
• Nginx: Serves static files directly, handles client connection buffering
• Docker Compose: Manages multi-container applications with service dependencies

Performance Specifications

• Gunicorn workers: (2 × CPU cores) + 1 starting formula
• Worker recycling: --max-requests 1000 prevents memory leak accumulation
• Database connections: Monitor with SELECT count(*) FROM pg_stat_activity;
• Memory monitoring: Use docker stats to catch resource growth early

Configuration Requirements

Security Settings (Production Django)

DEBUG = False
SECURE_BROWSER_XSS_FILTER = True
SECURE_CONTENT_TYPE_NOSNIFF = True
X_FRAME_OPTIONS = 'DENY'
SECURE_HSTS_SECONDS = 31536000
SECURE_HSTS_INCLUDE_SUBDOMAINS = True
SECURE_HSTS_PRELOAD = True

Database Configuration

healthcheck:
  test: ["CMD-SHELL", "pg_isready -U ${DB_USER}"]
  interval: 30s
  timeout: 10s
  retries: 5

Nginx Optimization

location /static/ {
    alias /home/app/web/staticfiles/;
    expires 1y;
    add_header Cache-Control "public, immutable";
}
client_max_body_size 100M;

Resource Limits and Scaling

Container Resource Constraints

deploy:
  resources:
    limits:
      memory: 512M
      cpus: '0.5'

PostgreSQL Tuning

environment:
  - POSTGRES_SHARED_PRELOAD_LIBRARIES=pg_stat_statements
  - POSTGRES_MAX_CONNECTIONS=100
  - POSTGRES_SHARED_BUFFERS=256MB

Critical Deployment Commands

Initial Setup Sequence

# Build with no cache (10+ minutes first time)
docker-compose -f docker-compose.prod.yml build --no-cache

# Start services
docker-compose -f docker-compose.prod.yml up -d

# Wait for database ready signal
docker-compose -f docker-compose.prod.yml logs -f db

# Run migrations
docker-compose -f docker-compose.prod.yml exec web python manage.py migrate --noinput

# Collect static files
docker-compose -f docker-compose.prod.yml exec web python manage.py collectstatic --no-input --clear

Backup and Recovery

# Database backup
docker-compose -f docker-compose.prod.yml exec db pg_dump -U $DB_USER $DB_NAME > backup_$(date +%Y%m%d_%H%M%S).sql

# Database restore
docker-compose -f docker-compose.prod.yml exec -T db psql -U $DB_USER $DB_NAME < backup.sql

Memory Leak Prevention

Worker Management

• Max requests: Set --max-requests 1000 to restart workers before memory accumulation
• Connection age: Avoid high CONN_MAX_AGE values that cause connection leaks
• Monitoring: Use docker stats to track memory growth patterns

Critical Indicators

• Memory usage above 90% indicates worker recycling issues
• Database connection count should drop when traffic decreases
• Worker process count should remain stable under load

Common Error Messages and Fixes

Database Errors

• FATAL: sorry, too many clients already → Check CONN_MAX_AGE and connection pooling
• ProgrammingError: relation "django_session" does not exist → Run migrations
• database files are incompatible with server → PostgreSQL version change requires data migration

Container Errors

• network django_default not found → Run docker network create django_default
• ECONNREFUSED → Service reachable but not accepting connections (still starting)
• KeyError: 'SECRET_KEY' → Missing environment variables in .env.prod

Performance Issues

• Slow responses → Increase worker count or timeout values
• Memory growth → Enable worker recycling and monitor connection leaks
• Static file 404s → Run collectstatic and verify volume mounting

Security Best Practices

Environment Variable Management

• Never commit .env.prod to version control
• Use docker secret create for sensitive data
• Generate strong SECRET_KEY with Django's get_random_secret_key()

Container Security

• Create non-root users: adduser --system app
• Switch user context: USER app
• Set proper directory ownership: chown -R app:app $APP_HOME

Network Security

• Implement HTTPS redirection
• Configure security headers in Nginx
• Use proper SSL certificate management

Monitoring and Maintenance

Health Checks

# Service status
docker-compose -f docker-compose.prod.yml ps

# Resource usage
docker stats

# Application logs
docker-compose -f docker-compose.prod.yml logs --tail=50

Database Maintenance

# Connection monitoring
docker-compose -f docker-compose.prod.yml exec db psql -U $DB_USER -d $DB_NAME -c "SELECT * FROM pg_stat_activity;"

# Database size check
docker-compose -f docker-compose.prod.yml exec db psql -U $DB_USER -d $DB_NAME -c "SELECT pg_size_pretty(pg_database_size('$DB_NAME'));"

Performance Thresholds

Critical Limits

• UI breaks: 1000+ spans make debugging distributed transactions impossible
• Memory per worker: 2GB indicates serious memory leaks requiring immediate worker recycling
• Database connections: Should not exceed max_connections setting under normal load
• Response time degradation: Timeout issues often indicate query optimization needs, not timeout increases

Scaling Indicators

• Worker CPU utilization consistently above 80%
• Database connection pool exhaustion
• Static file serving latency increasing
• Memory usage growth patterns over time

Troubleshooting Decision Tree

Container Won't Start

1. Check logs with docker-compose logs web
2. Verify environment variables in .env.prod
3. Confirm file permissions and line endings
4. Test service networking with ping between containers

Database Issues

1. Verify PostgreSQL health check passes
2. Check connection string service names
3. Confirm migrations ran successfully
4. Monitor connection counts and query performance

Static File Problems

1. Verify collectstatic execution
2. Check volume mounting in both services
3. Confirm Nginx configuration paths
4. Test cache headers and browser behavior

Performance Degradation

1. Monitor worker processes and memory usage
2. Check database query performance
3. Verify connection pooling configuration
4. Analyze resource utilization patterns

This guide provides actionable intelligence for production Django deployments, focusing on failure prevention, performance optimization, and operational reliability rather than basic setup instructions.