Fix network load balancing issues by verifying the balancer is up and listening, confirming backend health checks pass, matching the algorithm to your traffic pattern, aligning session persistence with application behaviour, and clearing firewall or DNS rules that block probes. Then re-test with synthetic traffic before restoring production load.
Symptoms
- One backend takes almost all requests while others sit idle or show near-zero connections.
- Users get logged out mid-session or lose shopping carts between page loads.
- Intermittent 502, 503 or connection reset errors under normal traffic volume.
- Health check dashboard flaps: backends toggle between healthy and unhealthy every few seconds.
- DNS returns the balancer VIP, but clients still hit a single origin directly.
Common Causes
Algorithm mismatched to workload
Round-robin on long-lived WebSocket or database sessions concentrates load on whichever server answered first. Least-connections or weighted routing usually fits better.
Broken or too-strict health checks
Probe hits a path that requires auth, or the timeout is shorter than the backend's warm-up. Servers get marked unhealthy and drop out of rotation.
Session persistence misconfigured
Sticky sessions disabled for stateful apps, or cookie name mismatched between balancer and app, causes logouts and cart loss on every request.
Firewall blocks probes or return traffic
ACLs permit client traffic but drop the balancer's health-check source IP, or asymmetric routing sends replies around the balancer.
DNS or VIP resolution issue
Stale A records, low TTL not honoured, or clients cached a single backend IP directly, bypassing the VIP entirely.
Step-by-Step Fix
- Confirm the load balancer process is running and listening
Check the service state first. On Linux with HAProxy or Nginx, run systemctl status and ss -tlnp to confirm the frontend port is bound. On Windows NLB, open the Network Load Balancing Manager and check cluster convergence. A silently crashed worker looks identical to a routing bug from the client side. - Read the balancer's own stats page before touching backends
HAProxy exposes /stats, Nginx Plus has a dashboard, F5 has the LTM Traffic Summary, and AWS ELB has CloudWatch RequestCount per target. Look at per-backend request counts and current sessions. If one node handles 90% of traffic, the problem is algorithm or persistence, not the servers themselves. - Test each backend directly, bypassing the VIP
Curl each backend's private IP on the app port from the balancer host itself. If a direct hit fails or is slow, the backend is the issue. If direct hits are healthy but the VIP is not, the problem lives in the balancer config, health check, or network path between them. - Review health check definition and thresholds
Match the probe to something the app actually serves cheaply, such as /healthz returning 200. Set interval, timeout and unhealthy-threshold so a brief GC pause does not evict a node. Watch the check flap counter for a few minutes after any change. - Pick an algorithm that matches session length
Short stateless HTTP tolerates round-robin. Long-lived connections such as SSH, RDP, WebSocket, gRPC streams or database pools need least-connections or least-response-time. Weighted variants help when backends have different CPU or memory sizes. - Align session persistence with the application
Stateful apps without shared session storage need stickiness: source-IP hash for TCP, or cookie-based affinity for HTTP. Confirm the cookie name and TTL match what the app expects. Better long-term fix is external session storage in Redis or a database, then persistence can be dropped. - Open firewall paths for probes and return traffic
Allow the balancer's health-check source range to reach every backend on the app port. Check that backends route replies via the balancer, not out a default gateway, or you will see asymmetric routing and connection resets. On cloud, verify security groups on both sides. - Validate DNS and client-side caching
Resolve the public hostname from an external client with dig or nslookup and confirm it returns the VIP, not a backend IP. Lower TTL before planned changes. If clients hardcoded a backend IP, fix the client config; DNS alone will not save you.
Common load balancing symptoms mapped to likely cause and first fix
| Symptom | Likely cause | First action |
|---|---|---|
| All traffic to one backend | Sticky sessions too aggressive or hash on small key space | Switch to least-connections or widen hash input |
| Users logged out randomly | No session persistence, no shared session store | Enable cookie affinity or move sessions to Redis |
| Backends flap healthy/unhealthy | Probe timeout shorter than app response time | Increase timeout and unhealthy-threshold |
| 502 or 503 under light load | Backend pool empty, all marked unhealthy | Fix probe path, then re-check backend service |
| Direct backend works, VIP fails | Firewall or routing between balancer and backend | Open probe source IP, check return path |
| Traffic bypasses balancer | Client cached backend IP or wrong DNS record | Correct A record, lower TTL, fix client config |
Prevention
- Alert on per-backend request-count skew, not just total error rate.
- Load-test after every balancer or app config change, not only at launch.
- Keep balancer firmware and TLS libraries patched on a fixed cadence.
- Document the health check contract so app teams do not accidentally break /healthz.
FAQ
Should I use round-robin or least-connections?
Round-robin is fine for short, stateless HTTP requests of roughly equal cost. Least-connections is better whenever sessions vary in length, such as APIs with slow endpoints, WebSocket, RDP or database traffic. If your backends have different sizes, add weights so the larger nodes receive proportionally more requests.
Why do my users get logged out when I enable the load balancer?
The app stores session state in local memory, so each request may land on a different backend that has no record of the user. Fix it either by enabling session persistence at the balancer, usually via cookie affinity, or by moving session state into a shared store like Redis. The shared-store approach scales better and survives node loss.
How do I stop backends from flapping in and out of the pool?
Flapping usually means the health check is too strict for real backend behaviour. Increase the probe timeout so brief GC pauses or disk flushes do not fail a check, and raise the unhealthy-threshold to require several consecutive failures. Make sure the probe path is cheap and does not depend on downstream services that can fail independently.
Need a second pair of eyes on your load balancer setup? Contact our network team for a configuration review.