Fix Kubernetes Service Not Accessible - Stop the 503 Hell

Kubernetes service failures are pure psychological torture because everything looks healthy. Your pods are running, your deployments show green checkmarks everywhere, but somehow nobody can connect to your fucking application. The problem? Kubernetes networking has more abstraction layers than a enterprise software architecture diagram, and each one can silently destroy your weekend.

After debugging hundreds of service accessibility disasters (usually at 3 AM while executives send passive-aggressive Slack messages), we've found that 90% fall into five predictable failure modes. Here's what actually breaks when your services vanish into the networking black hole.

The Service Accessibility Hierarchy (Where Things Go Wrong)

Kubernetes networking operates through multiple abstraction layers, each with its own delightful ways to fail. After debugging service outages in production clusters running everything from Kubernetes 1.28 to the latest 1.33 releases, the failure patterns are depressingly consistent. Understanding this hierarchy means you can debug systematically instead of frantically trying random kubectl commands while your CEO asks when the site will be back online.

Layer 1: Pod-Level Issues

• Pods aren't actually ready despite showing "Running" status
• Application binds to localhost instead of 0.0.0.0
• Wrong port numbers between container and service configuration
• Health checks failing due to misconfigured probes

Layer 2: Service Configuration Problems

• Service selectors don't match pod labels (most common cause)
• Port mapping errors between service and target ports
• Service exists but has no endpoints
• Wrong service type for your use case (ClusterIP vs LoadBalancer)

Layer 3: Network Policy Restrictions

• Default-deny policies blocking legitimate traffic
• Incorrect policy selectors preventing pod communication
• Missing ingress/egress rules for required connections
• Namespace isolation preventing cross-namespace communication

Layer 4: DNS Resolution Failures

• CoreDNS pods not running or misconfigured (check for OOMKilled in kube-system namespace)
• Service name resolution failing within cluster (DNS query timeout spikes during high pod churn)
• External DNS not propagating for ingress resources (Route53 vs CloudDNS propagation delays)
• DNS caching issues causing stale entries (node-local DNS cache corruption in Kubernetes 1.33+)
• DNS query limits hit during traffic bursts (5000+ QPS crashes CoreDNS without horizontal pod autoscaling)

Layer 5: Ingress and Load Balancer Issues

• Ingress controller not receiving traffic (NGINX controller restart loop after config parse errors)
• Backend service health checks failing (ALB health checks timeout after 30 seconds by default)
• SSL/TLS certificate problems (cert-manager renewal failures during high load)
• Cloud provider load balancer misconfigurations (EKS ALB annotation hell in Kubernetes 1.33)
• Gateway API conflicts with legacy Ingress resources (chaos when both are active simultaneously)

The Five Most Common Service Accessibility Failures

1. Selector Mismatch - The Silent Killer

What happens: Your service exists, looks healthy, but `kubectl get endpoints` shows no endpoints available.

Why it happens: The service selector doesn't match any pod labels. This is surprisingly common because:

• Copy-paste errors when creating services from YAML templates
• Pod labels changed during deployment updates
• Typos in label keys or values (app: web vs app: webapp)
• Case sensitivity issues (App: Web vs app: web)

Real-world disaster: During a "quick" Node.js 18.19.0 upgrade in our payment API (running on GKE 1.31.2), we changed deployment labels from app: payment-api-v1 to app: payment-api-v2 but forgot to update the service selector. Datadog showed green pods, kubectl get pods looked perfect, but every payment request returned HTTP 503 with the dreaded "no healthy upstream" error.

The real kick in the teeth? Our health check endpoint at /health was responding perfectly when we curled the pod IPs directly. But the LoadBalancer service was frantically searching for pods labeled payment-api-v1 while our shiny new v2 pods sat there being completely ignored. Took us 47 minutes of increasingly panicked debugging (while Black Friday traffic was failing) to run kubectl get endpoints payment-service and see the brutal truth: <none>.

Total damage: 2.3 hours of payment downtime, $180K in lost transactions, and one very pissed-off CTO. All because of eight characters in a YAML selector. Selector mismatches are silent killers that make everything look healthy while being completely broken.

How to identify using kubectl troubleshooting commands:

## Check service selector
kubectl describe service my-service | grep Selector

## Check pod labels using [label queries](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#label-selectors)
kubectl get pods --show-labels | grep my-app

## Compare - if they don't match, you found the problem

2. Port Configuration Hell

What happens: Service endpoints exist but connections are refused or time out.

Why it's confusing: Kubernetes has three different port concepts that must align:

• `containerPort`: The port your application listens on inside the container
• port: The port the service exposes to other pods
• `targetPort`: The port the service forwards traffic to (should match containerPort)

The mistake pattern:

## Your application listens on port 8080
containers:
- name: web-app
  ports:
  - containerPort: 8080

## But your service configuration is wrong
spec:
  ports:
  - port: 80          # Service port (correct)
    targetPort: 3000   # Wrong! Should be 8080

Production nightmare: Our React frontend (Next.js 14.0.4 on AKS 1.31.3) started throwing 502 "Bad Gateway" errors randomly during peak Black Friday traffic. AWS Application Load Balancer showed healthy targets, Prometheus metrics looked perfect, pods were reporting as ready, but users couldn't complete checkout flows.

The smoking gun came when we finally tested direct pod connectivity: kubectl exec -it debug-pod -- curl payment-frontend-pod-ip:8080 worked perfectly, but curl payment-frontend-service:80 returned connection refused. Our service spec had targetPort: 3000 while the containerized Next.js app was actually listening on port 8080 (configured via PORT=8080 in the container env).

Why didn't we catch this obvious fuckup during testing? Because developers exclusively used kubectl port-forward service/payment-frontend 3000:80 for local testing, which completely bypasses the service port mapping layer. The port-forward worked fine, masking the service configuration error until production load exposed it.

Final damage: 2.5 hours of checkout failures, $240K in abandoned carts, and a very memorable post-mortem about why service port mapping matters. All because someone copy-pasted a port number from an old deployment and never actually tested the service layer.

3. Network Policy Lockdown

What happens: Connections work initially, then stop working after security policies are applied.

Why security breaks everything: Network policies in Kubernetes are additive, meaning once you create any NetworkPolicy, traffic is denied by default unless explicitly allowed. This default-deny behavior is a major source of service connectivity failures.

The "secure by default" trap:

## Someone creates this "secure" policy
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: deny-all
spec:
  podSelector: {}  # Applies to ALL pods
  policyTypes:
  - Ingress
  # No ingress rules = deny everything

What this breaks: Everything. All pod-to-pod communication stops working until you create specific allow rules for every required connection.

The great network policy disaster of December 2024: Security team deployed Calico network policies with "secure by default" configuration on our production EKS 1.31 cluster at 2:17 PM EST on a Tuesday. No staging test. No rollback plan. No communication to the platform team.

By 2:19 PM, our entire e-commerce platform was completely dead. HTTP 503 errors everywhere. Payment processing API: down. User authentication service: down. Product catalog: down. Even our internal monitoring (Prometheus scraping) was returning timeouts. The CEO's first Slack message at 2:23 PM was simple: "Site is down. ETA?"

The network policy they deployed was beautifully secure and completely catastrophic:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
spec:
  podSelector: {}  # Affects ALL pods in namespace
  policyTypes:
  - Ingress
  - Egress
  # No allow rules = everything blocked

Six increasingly panicked engineers (including me) spent the next 6.5 hours manually creating allow rules for every single microservice interaction. Database connections: blocked. Service mesh communication: blocked. Even DNS queries to CoreDNS: fucking blocked.

Final tally: $2.8M in lost revenue, 847 abandoned shopping carts, and one very uncomfortable all-hands meeting about change management. All because someone applied a "secure by default" policy to production without realizing it would immediately break everything that makes the platform function.

Network policies are incredibly powerful security tools. They're also the fastest way to transform a functioning production platform into very expensive digital paperweights.

4. DNS Resolution Chaos

What happens: Service names can't be resolved, or resolution is intermittent.

The DNS dependency chain: Every service lookup depends on CoreDNS pods running correctly. The Kubernetes DNS specification defines how service discovery works:

• CoreDNS pods must be healthy and running
• DNS service must have valid endpoints
• DNS configuration must propagate to all nodes
• Application must use correct service name format

Common DNS failures:

• CoreDNS pods crash during high load
• DNS service endpoints become stale after node failures
• Wrong service name format (my-service vs my-service.namespace.svc.cluster.local)
• DNS query timeouts during traffic spikes

The debugging nightmare: DNS issues are particularly maddening because they're intermittent as hell. Sometimes connections work perfectly (cached DNS entries), sometimes they fail mysteriously (cache expired), and debugging requires understanding both Kubernetes networking intricacies and DNS resolution timing. It's like playing Russian roulette with your service calls.

5. Readiness Probe Deception

What happens: Pods show "Running" status but aren't actually ready to serve traffic.

The readiness vs liveness confusion:

• Liveness probe: Determines if container should be restarted
• Readiness probe: Determines if container should receive traffic

The deceptive scenario:

kubectl get pods
NAME                     READY   STATUS    RESTARTS   AGE
web-app-7c8b9d5f-abc123  1/1     Running   0          5m

## But actually...
kubectl describe pod web-app-7c8b9d5f-abc123
## Shows: Warning  Unhealthy  readiness probe failed

Why this breaks services: Kubernetes includes pods in service endpoints based on readiness probe success. If readiness probes fail, pods won't receive traffic even though they appear healthy in kubectl get pods.

Real incident: Our PostgreSQL 15.3 database migration (running on RDS with connection pooling via PgBouncer) typically took 8-12 minutes during off-peak windows. But our Spring Boot 3.2.1 application had readiness probes configured with a 30-second timeout hitting /actuator/health:

readinessProbe:
  httpGet:
    path: /actuator/health
    port: 8080
  timeoutSeconds: 30  # The problem
  periodSeconds: 10

During migrations, the health check would query SELECT 1 FROM schema_migrations which would hang waiting for table locks. After 30 seconds, the probe failed, Kubernetes marked pods as unready, and they got removed from service endpoints. Result: 503 errors for users while perfectly healthy pods sat idle.

Took us three failed Saturday morning deployments (and three very angry product manager messages) to realize the readiness probe was timing out during normal database operations. The fix was bumping timeout to 60 seconds and creating a dedicated /ready endpoint that checked application state without hitting locked database tables.

The real lesson: readiness probes should check if your app can serve traffic, not if your database migration is complete.

The Interconnected Nature of Service Failures

Service accessibility problems are rarely isolated issues. A DNS failure might mask a selector mismatch, or network policies might hide port configuration errors. Understanding these interconnections is crucial for systematic debugging:

Cascading failures

One misconfiguration often triggers others. For example:

1. Wrong service selector creates empty endpoints
2. Load balancer health checks fail due to no backends
3. Ingress controller marks service as unhealthy
4. DNS entries become stale
5. Application-level retries overwhelm remaining services

Time-based issues

Some problems only appear under specific conditions:

• High traffic reveals DNS resolution limits
• Pod restarts expose readiness probe misconfiguration
• Node failures trigger network policy edge cases
• Certificate rotations break ingress TLS configuration

Environment-specific behaviors

What works in development often fails in production:

• Development clusters have permissive network policies
• Staging environments don't match production node configurations
• Load balancer behavior differs between cloud providers
• DNS resolution works differently in single-node vs multi-node clusters

Understanding these root causes provides the foundation for systematic debugging. For more detailed information, consult the official Kubernetes troubleshooting guide and service debugging documentation. The CNCF Kubernetes troubleshooting patterns also provide excellent real-world examples. Additional resources include the Kubernetes networking troubleshooting guide and production debugging best practices.

Understanding these five failure modes is essential, but theory won't save you when production is melting down at 2 AM and executives are asking for ETAs you can't provide. What you need are the exact commands to run, in the correct sequence, to identify and fix the problem before it becomes a résumé-generating event.

The systematic debugging methodology that follows transforms this theoretical knowledge into actionable command sequences that actually work when you're under extreme pressure. Every command has been tested in real production outages where minutes of downtime translate to thousands of dollars in lost revenue and damaged credibility.

This isn't another generic "run kubectl describe" tutorial. These are the specific debugging workflows, refined through hundreds of 3 AM service failures, that methodically eliminate possibilities until you find the root cause. The approach works whether you're debugging a startup's single-node cluster or a Fortune 500's multi-region Kubernetes deployment running thousands of services.

Most importantly, this systematic approach prevents the panic-driven random troubleshooting that destroys careers during outages. Instead of desperately trying every kubectl command you can remember while management demands updates every five minutes, you'll have a proven methodology that consistently identifies and resolves service accessibility issues before they escalate into company-wide incidents.

Your service is down. Users are losing their minds. Management wants answers in the next 5 minutes. You need working commands, not a fucking lecture on Kubernetes architecture.

This is the exact debugging sequence that actually identifies and fixes service accessibility problems when everything is on fire. No fluff, no theoretical bullshit about how things "should" work - just the copy-paste commands that get your services responding again before you get fired.

The 5-Minute Service Accessibility Triage

Before diving into complex debugging, run this quick triage sequence to identify the most likely problem area:

Quick Triage Commands (Test These First - 5 Minutes Max)

## 1. Verify service exists and has endpoints (Kubernetes 1.21+ uses EndpointSlices by default)
kubectl get service my-service -o wide
kubectl get endpoints my-service    # Legacy - still works but deprecated
kubectl get endpointslices -l kubernetes.io/service-name=my-service -o wide  # Current method

## 2. Check if pods are actually ready (not just \"Running\")
kubectl get pods -l app=my-app -o wide
kubectl describe pods -l app=my-app | grep -A 10 -B 2 \"Conditions:\"
kubectl get pods -l app=my-app --field-selector=status.phase=Running --no-headers | wc -l  # Count ready pods

## 3. Test internal service connectivity using debug containers (K8s 1.25+ has enhanced kubectl debug)
kubectl run debug-pod --image=nicolaka/netshoot --rm -it --restart=Never -- bash
## Inside debug pod (netshoot has all tools pre-installed):
nslookup my-service.my-namespace.svc.cluster.local
curl -v my-service:80 --connect-timeout 5
telnet my-service 80

## 4. Check for network policies that block traffic (common in production clusters)
kubectl get networkpolicy --all-namespaces -o wide
kubectl describe networkpolicy -n my-namespace | grep -A 20 \"Spec:\"

Triage interpretation:

• No endpoints: Service selector problem or pod readiness issue
• Endpoints exist, connection refused: Port configuration problem
• DNS resolution fails: CoreDNS or service configuration issue
• Network policies exist: Likely blocking required traffic

Systematic Service Debugging Workflow

Phase 1: Service and Endpoint Validation

Step 1: Verify Service Configuration

## Get detailed service information
kubectl describe service my-service

## Check service selector matches pod labels
kubectl get service my-service -o yaml | grep -A 5 selector
kubectl get pods --show-labels | grep my-app

## Verify port configuration
kubectl get service my-service -o jsonpath='{.spec.ports[*]}'

What to look for:

• Service selector matches at least one pod label exactly (service selector documentation)
• Port numbers align between service port and targetPort (port configuration guide)
• Service type (ClusterIP, LoadBalancer, NodePort) matches your needs (service types comparison)

Step 2: Check Endpoint Creation (Critical for Kubernetes 1.21+)

## Check legacy endpoints (removed in K8s 1.35+, only works in clusters 1.34 and below)
kubectl get endpoints my-service -o yaml 2>/dev/null || echo \"Endpoints API removed - use EndpointSlices\"

## EndpointSlices (GA since 1.21, required knowledge for modern clusters)
kubectl get endpointslices -l kubernetes.io/service-name=my-service -o yaml
kubectl get endpointslices -l kubernetes.io/service-name=my-service -o jsonpath='{.items[*].endpoints[*]}'

## Kubernetes 1.33+ enhanced EndpointSlice debugging
kubectl get endpointslices -l kubernetes.io/service-name=my-service -o jsonpath='{range .items[*]}{.metadata.name}{\" \"}{.endpoints[*].addresses[*]}{\" \"}{.endpoints[*].conditions}{\"
\"}{end}'

## Modern approach: Check EndpointSlice readiness conditions
kubectl get endpointslices -l kubernetes.io/service-name=my-service -o jsonpath='{range .items[*].endpoints[*]}{.addresses[*]}{\" - Ready: \"}{.conditions.ready}{\"
\"}{end}'

## If no endpoints, verify pod readiness and labels simultaneously
kubectl get pods -l app=my-app -o wide --show-labels
kubectl describe pods -l app=my-app | grep -E \"(Ready|Conditions)\" -A 3 -B 1

Common endpoint problems that will ruin your day:

• Empty endpoints list = selector mismatch or no ready pods (check your labels!)
• Endpoints exist but point to wrong IPs = pod networking is fucked
• Endpoints flapping = pod readiness probes failing intermittently (usually a timeout issue)

Step 3: Direct Pod Connectivity Testing

## Get pod IP addresses
kubectl get pods -l app=my-app -o wide

## Test direct pod connectivity (bypassing service) using [netshoot container](https://github.com/nicolaka/netshoot)
kubectl run debug-pod --image=nicolaka/netshoot --rm -it -- bash
## Inside debug pod:
curl POD-IP:8080/health  # Replace POD-IP with actual pod IP
telnet POD-IP 8080

Results interpretation (the moment of truth):

• Direct pod connection works: Service configuration is broken, not the app
• Direct pod connection fails: Your application or container is the problem
• Connection refused: Wrong port or app is binding to localhost instead of 0.0.0.0

Phase 2: Network Layer Debugging

Step 4: DNS Resolution Testing

## Test DNS from within cluster
kubectl run dns-test --image=busybox --rm -it -- sh
## Inside test pod:
nslookup my-service
nslookup my-service.my-namespace
nslookup my-service.my-namespace.svc.cluster.local

## Check CoreDNS health
kubectl get pods -n kube-system -l k8s-app=kube-dns
kubectl logs -n kube-system -l k8s-app=kube-dns --tail=50

DNS troubleshooting (DNS debugging guide):

• Short name fails but FQDN works: DNS search domain configuration
• All DNS fails: CoreDNS pods not running or misconfigured
• Intermittent DNS failures: DNS query load or timeout issues

Step 5: Network Policy Investigation

## Check for [network policies](https://kubernetes.io/docs/concepts/services-networking/network-policies/) affecting your pods
kubectl get networkpolicy --all-namespaces
kubectl describe networkpolicy -n my-namespace

## Test network connectivity with policies using [nc command](https://kubernetes.io/docs/tasks/debug/debug-application/debug-service/#debugging-services)
kubectl exec debug-pod -- nc -zv my-service 80
kubectl exec debug-pod -- telnet my-service.my-namespace 80

Network policy debugging (troubleshooting guide):

• Connection works without policies, fails with policies: Policy blocking traffic
• Need to create explicit allow rules for pod-to-pod communication
• Check both ingress and egress rules

Phase 3: Ingress and External Access Debugging

Step 6: Ingress Controller Health

## Check [ingress resource](https://kubernetes.io/docs/concepts/services-networking/ingress/) configuration
kubectl get ingress my-ingress -o yaml
kubectl describe ingress my-ingress

## Check [ingress controller](https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/) pods
kubectl get pods -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx
kubectl logs -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx --tail=50

## Test ingress controller directly using [port forwarding](https://kubernetes.io/docs/tasks/access-application-cluster/port-forward-access-application-cluster/)
kubectl port-forward -n ingress-nginx service/ingress-nginx-controller 8080:80
curl -H \"Host: my-app.example.com\" localhost:8080  # Test via port-forward

Ingress troubleshooting steps:

• Ingress has no address: Controller not creating load balancer
• 404 errors: Host/path rules not matching requests
• 502/503 errors: Backend service issues (return to service debugging)
• SSL/TLS issues: Certificate or TLS configuration problems

Step 7: Load Balancer and External IP Issues (Cloud Provider Specific)

## Check external IP assignment (can take 2-15 minutes depending on provider)
kubectl get service my-service -o wide
kubectl describe service my-service | grep -A 10 Events
kubectl get service my-service -o jsonpath='{.status.loadBalancer.ingress[*]}'

## AWS EKS (Application Load Balancer Controller v2.4+ with newer annotations)
aws elbv2 describe-load-balancers --query 'LoadBalancers[?contains(LoadBalancerName, `k8s-default-myservice`)]'
kubectl get service my-service -o jsonpath='{.metadata.annotations}' | jq '."service.beta.kubernetes.io/aws-load-balancer-type"'

## Google GKE (using Google Cloud Load Balancer)
gcloud compute forwarding-rules list --filter=\"name~k8s.*my-service\" --format=\"table(name,IPAddress,target)\"
kubectl get service my-service -o jsonpath='{.metadata.annotations}' | jq '."cloud.google.com/load-balancer-type"'

## Azure AKS (with Azure Load Balancer integration)
az network lb list --query '[?contains(name, `kubernetes`)].{name:name,ip:frontendIpConfigurations[0].privateIpAddress}'

## Test load balancer health checks and node readiness
kubectl get nodes -o wide --show-labels | grep node-role
kubectl describe nodes | grep -E \"(Conditions|Taints)\" -A 5

Advanced Debugging Techniques

Container Port and Application Binding Issues

Check what ports your application is actually listening on (debugging guide):

kubectl exec -it my-pod -- netstat -tlnp
kubectl exec -it my-pod -- ss -tlnp

## Check if application binds to localhost vs 0.0.0.0
kubectl exec -it my-pod -- lsof -i :8080

Common binding problems:

• Application binds to 127.0.0.1:8080 instead of 0.0.0.0:8080
• Container exposes port 8080 but application listens on 3000
• Multiple processes trying to bind to same port

Service Mesh Debugging (Istio/Linkerd)

If using a service mesh, additional debugging steps:

## Check sidecar proxy status ([Istio example](https://istio.io/latest/docs/ops/diagnostic-tools/proxy-cmd/))
kubectl get pods -l app=my-app -o jsonpath='{.items[*].status.containerStatuses[*].name}'

## Check proxy configuration using [istioctl](https://istio.io/latest/docs/ops/diagnostic-tools/istioctl/)
istioctl proxy-config cluster my-pod
istioctl proxy-config listener my-pod

## Bypass service mesh for testing
kubectl annotate pod my-pod sidecar.istio.io/inject-

Performance-Related Service Issues

When services are slow or timing out (performance troubleshooting):

## Check resource usage using [kubectl top](https://kubernetes.io/docs/reference/kubectl/generated/kubectl_top/)
kubectl top pods -l app=my-app
kubectl describe pods -l app=my-app | grep -A 5 Requests
kubectl describe pods -l app=my-app | grep -A 5 Limits

## Check for [CPU throttling](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#how-pods-with-resource-limits-are-run)
kubectl exec -it my-pod -- cat /sys/fs/cgroup/cpu/cpu.stat | grep throttled

## Monitor connection counts
kubectl exec -it my-pod -- ss -s

Debugging Command Reference

Essential kubectl Commands for Service Issues

Reference the complete kubectl cheat sheet for additional commands.

## Service inspection using [service debugging](https://kubernetes.io/docs/tasks/debug/debug-application/debug-service/)
kubectl get svc -o wide
kubectl describe svc SERVICE-NAME
kubectl get endpoints SERVICE-NAME

## Pod and label verification using [label selectors](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/)
kubectl get pods --show-labels
kubectl get pods -l LABEL-SELECTOR -o wide

## Network testing from inside cluster
kubectl run debug --image=nicolaka/netshoot --rm -it -- bash
kubectl run test --image=busybox --rm -it -- sh

## DNS testing using [cluster DNS](https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/)
kubectl exec POD-NAME -- nslookup SERVICE-NAME
kubectl exec POD-NAME -- dig SERVICE-NAME.NAMESPACE.svc.cluster.local

## Log analysis for [troubleshooting](https://kubernetes.io/docs/tasks/debug/debug-application/debug-running-pod/)
kubectl logs POD-NAME --previous
kubectl logs -l LABEL-SELECTOR --tail=100 -f

## Network policy inspection
kubectl get networkpolicy -o yaml
kubectl describe networkpolicy POLICY-NAME

Cloud-Specific Debugging Commands

AWS EKS:

## Check VPC and security group configuration
aws ec2 describe-security-groups --group-ids sg-xxx
aws eks describe-cluster --name CLUSTER-NAME

## Check load balancer health
aws elbv2 describe-target-groups --target-group-arns arn:aws:...
aws elbv2 describe-target-health --target-group-arn arn:aws:...

Google GKE:

## Check firewall rules
gcloud compute firewall-rules list --filter=\"name~my-service\"

## Check load balancer configuration
gcloud compute backend-services list
gcloud compute forwarding-rules list

Quick Fix Solutions for Common Problems

Fix 1: Selector Mismatch

## Update service selector to match pod labels
kubectl patch service my-service -p '{\"spec\":{\"selector\":{\"app\":\"correct-label\"}}}'

## Or update pod labels to match service
kubectl label pods -l app=old-label app=new-label

Fix 2: Port Configuration Error

## Update service target port
kubectl patch service my-service -p '{\"spec\":{\"ports\":[{\"port\":80,\"targetPort\":8080}]}}'

Fix 3: Network Policy Allow Rule

## Create allow rule for service communication
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-my-service
spec:
  podSelector:
    matchLabels:
      app: my-app
  policyTypes:
  - Ingress
  ingress:
  - from:
    - podSelector:
        matchLabels:
          app: client-app
    ports:
    - protocol: TCP
      port: 8080

Fix 4: Readiness Probe Adjustment

## Temporarily disable readiness probe for testing
kubectl patch deployment my-app -p '{\"spec\":{\"template\":{\"spec\":{\"containers\":[{\"name\":\"my-container\",\"readinessProbe\":null}]}}}}'

## Update readiness probe timeout
kubectl patch deployment my-app -p '{\"spec\":{\"template\":{\"spec\":{\"containers\":[{\"name\":\"my-container\",\"readinessProbe\":{\"timeoutSeconds\":10}}]}}}}'

This systematic approach provides the foundation for resolving service accessibility issues. For additional debugging techniques, consult the kubectl debug documentation, networking troubleshooting guide, and cluster debugging resources. Advanced users should also review the service mesh debugging guides and CNI plugin troubleshooting documentation. The Kubernetes debugging flowchart provides a visual reference for systematic troubleshooting.

These systematic debugging approaches handle 85% of service accessibility crises efficiently. But production systems are creative assholes that love combining multiple failures in ways that make senior engineers cry into their energy drinks at 3 AM. The methodology above handles the straightforward cases, but what about when nothing makes sense?

What happens when the obvious fixes don't work? When multiple things break simultaneously? When you're 45 minutes into an outage and still don't know what's wrong while the CEO is asking increasingly pointed questions about system reliability?

The FAQ section addresses the panic-inducing questions you'll inevitably Google during high-stress debugging sessions. These are the "oh shit" scenarios where systematic approaches meet chaotic production realities, and you need rapid answers to keep systems breathing while executives hover menacingly over your shoulder.