cert-manager: Kubernetes Certificate Management - AI Reference

Core Value Proposition

Problem Solved: Eliminates manual SSL certificate management that causes production outages during certificate expiration
Critical Failure Scenario: SSL certificate expiration during high-traffic periods (Black Friday example: 4-hour e-commerce outage, direct revenue impact)
Automation Benefit: Prevents 3 AM paging incidents from expired certificates

Technical Specifications

Core Components

• Certificate Resources: Kubernetes custom resources defining domain certificate requirements
• Issuer/ClusterIssuer Resources: Certificate Authority configuration (ClusterIssuer = cluster-wide, Issuer = namespace-scoped)
• CertificateRequest Resources: X.509 standard certificate signing requests (auto-generated, rarely manually managed)

Resource Requirements (Production)

resources:
  limits:
    cpu: 200m      # Default 100m insufficient under load
    memory: 256Mi   # Default 128Mi causes OOM during mass renewals
  requests:
    cpu: 50m
    memory: 64Mi

Challenge Methods

Method	Use Case	Failure Mode	Requirements
HTTP-01	Public services	Ingress controller conflicts, firewall blocks port 80	Public domain, ingress controller
DNS-01	Wildcard certs, internal services	DNS propagation delays (GoDaddy: 30+ minutes)	DNS API access

Critical Configuration

Production-Ready Installation

helm repo add jetstack https://charts.jetstack.io
helm install cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --set installCRDs=true  # CRITICAL: Prevents webhook failures

Multi-Issuer Setup

• Let's Encrypt Production: Public certificates, free, 90-day lifecycle
• Let's Encrypt Staging: Testing environment, avoids rate limits
• Internal CA: Enterprise PKI integration
• HashiCorp Vault: Internal certificate management

DNS-01 Configuration (AWS Route53 Example)

spec:
  acme:
    solvers:
    - dns01:
        route53:
          region: us-east-1
          # Use IRSA instead of hardcoded keys for security

Critical Failure Modes

Rate Limiting (Let's Encrypt)

• Limit: 50 certificates per domain per week
• Consequence: 3-7 day lockout during mass renewals
• Prevention: Use staging environment for testing, plan certificate requests

Resource Exhaustion

• Scenario: Mass certificate renewal causes cert-manager pod crashes
• Root Cause: Default resource limits too low (100m CPU, 128Mi memory)
• Solution: Increase limits to 200m CPU, 256Mi memory minimum

DNS Propagation Delays

• Common Providers: GoDaddy (30+ minutes), Namecheap (slow), Cloudflare (fast)
• Impact: DNS-01 challenge timeouts, renewal failures
• Mitigation: Increase webhook timeout to 30 seconds

Webhook Timeout Issues

webhook:
  timeoutSeconds: 30  # Default 10s too short for DNS delays

Monitoring and Alerting

Essential Prometheus Metrics

• cert_manager_certificate_expiration_timestamp_seconds: Certificate expiration tracking
• cert_manager_certificate_renewal_timestamp_seconds: Renewal success monitoring
• cert_manager_acme_client_request_count: ACME API request monitoring

Critical Alert (7-day expiration warning)

- alert: CertManagerCertificateExpirySoon
  expr: cert_manager_certificate_expiration_timestamp_seconds - time() < 86400 * 7
  for: 1h
  annotations:
    summary: "Certificate expires in less than 7 days"

Technology Comparison Matrix

Solution	Setup Complexity	Failure Support	CA Support	Automation	Multi-cluster	Vendor Lock-in
cert-manager	Medium	Good docs/community	Universal	Full	Yes	None
Traefik Built-in	Low	Community forums	Let's Encrypt focus	Limited	Per-cluster	Traefik
Manual ACME Scripts	High	Self-support	Let's Encrypt	Cron-based	Custom tooling	None
Cloud Provider	Low	Paid support	Provider-specific	Partial	Per-cloud	High
Vault PKI	High	Enterprise support	Internal only	Manual approval	Complex	HashiCorp

Common Troubleshooting

HTTP-01 Challenge Failures

Debug Commands:

kubectl describe certificate <name>
kubectl describe certificaterequest <name>
kubectl describe challenge <name>

Root Causes:

• Load balancer not publicly accessible
• Incorrect ingress.class annotation
• Firewall blocking port 80
• Multiple ingress controllers conflict
• CloudFlare proxy mode interference

DNS-01 Challenge Failures

Root Causes:

• Invalid DNS API credentials
• Hosted zone mismatch
• DNS propagation delays (provider-specific)
• Missing IAM permissions for DNS record creation
• DNS provider API rate limiting

Security Considerations

Production Hardening

• Disable HTTP-01 solver if only using DNS-01: global.disableHTTP01Solver: true
• Use IAM Roles for Service Accounts (IRSA) instead of hardcoded AWS keys
• Implement certificate approval workflows with approver-policy for compliance
• Consider CSI driver for ephemeral certificates in high-security environments

Resource Planning

Scale Considerations

• Normal operation: 50MB RAM, 10m CPU per pod
• Mass renewal periods: Resource requirements spike significantly
• High availability: Run multiple replicas (replicaCount: 2)
• Multi-cluster: Use ClusterIssuer for organization-wide policies

Cost Factors

• Let's Encrypt certificates: Free
• DNS API costs: Route53 charges per query for DNS-01
• Compute resources: Minimal in normal operation
• Commercial CA certificates: Variable pricing
• Operational time: Significantly reduced vs manual management

Migration Strategy

From Manual to Automated

1. Install cert-manager alongside existing certificates
2. Configure ClusterIssuer for existing CA
3. Create Certificate resources for new services
4. Migrate existing services incrementally
5. Decommission manual certificate scripts

Critical: Never migrate all certificates simultaneously - gradual migration prevents production impact

Known Limitations

Let's Encrypt Constraints

• Public domains only (no private/internal domains)
• Rate limits enforce weekly planning requirements
• ACME protocol dependencies on external validation

DNS Provider Reliability

• DNS propagation inconsistency across providers
• API reliability varies significantly
• Some providers have extended propagation delays

Kubernetes Dependencies

• Requires functional ingress controller for HTTP-01
• Webhook validation adds cluster dependency
• etcd storage for certificate keys (unless using CSI driver)