What Is Load Balancing? How It Works, 6 Algorithms and Load Balancer vs Reverse Proxy in 2026

Your Django API runs fine on one EC2 box until the Monday dispatch rush, and then response times triple and someone in sales asks why the app is slow again. So you launch a second server. Now you have a different problem: nothing is sending traffic to it. That gap, between having capacity and using it, is the whole job of a load balancer.

Keep one picture in your head throughout. A two person platform team at a Pune logistics SaaS runs a Django API peaking near 400 requests per second between 9 and 11 AM: two app servers, one Postgres primary, one deploy a week. That is the size at which this stops being theory.

How Load Balancing Actually Works

A load balancer owns the public address your users hit, so nothing reaches your app servers directly. It picks a backend for each request in four steps, and only the third is the part people talk about:

1. Accept the connection. Your DNS name resolves to the balancer, not a server, which is why swapping backends needs no client side change.
2. Consult the health table. Backends that failed their last probe are not candidates.
3. Apply the algorithm, chosen from the healthy list only.
4. Proxy the response back. The client never learns which server served it unless you leak that in a header.

Step two causes outages. A balancer with a perfect algorithm and a broken health check sends every request to a dead box.

The worked example: three backends on your laptop

Do this before touching a cloud console. Start three trivial backends, each serving one file naming its port:

mkdir -p /tmp/lb/{8001,8002,8003}
for p in 8001 8002 8003; do
  echo "served by $p" > /tmp/lb/$p/index.html
  (cd /tmp/lb/$p && python3 -m http.server $p) &
done

Now put Nginx in front. The upstream block is the whole idea: a named pool plus one directive picking the algorithm. least_conn has been in Nginx since 1.3.1, per the official ngx_http_upstream_module reference.

upstream dispatch_api {
    least_conn;
    server 127.0.0.1:8001 weight=2 max_fails=2 fail_timeout=10s;
    server 127.0.0.1:8002 max_fails=2 fail_timeout=10s;
    server 127.0.0.1:8003 max_fails=2 fail_timeout=10s backup;
}

server {
    listen 8080;
    location / {
        proxy_pass http://dispatch_api;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_connect_timeout 2s;
        proxy_next_upstream error timeout http_502 http_503;
    }
}

Reload Nginx, then prove the distribution rather than trusting the config:

for i in $(seq 1 20); do curl -s localhost:8080/ ; done | sort | uniq -c

Expect a two to one split between 8001 and 8002 because of weight=2, and nothing from 8003, since backup only takes traffic when the others are down. Now kill the 8001 process and rerun the loop. Traffic moves within one request, because proxy_next_upstream retries on a connection error instead of handing the user a 502. That directive is the difference between a blip and an incident.

The 6 Load Balancing Algorithms, and Which to Actually Pick

So which do you actually pick? For the Pune team, least connections, because their export endpoint is fifty times slower than their reads and round robin parks a long export behind a short one. That is the whole analysis. If your durations are uniform, round robin is fine and you should stop thinking about it. The trade off I am accepting is real: least connections makes distribution harder to read on a graph at 2 AM, since the split is no longer a flat line. I will take an ugly graph over a queued export.

Load Balancer vs Reverse Proxy: The Difference That Trips People Up

Every load balancer is a reverse proxy. Not every reverse proxy is a load balancer. The confusion survives because Nginx, HAProxy, Envoy and Traefik all do both jobs, so the same config file is either one depending on how many servers you listed.

Care about this for buying decisions. If your problem is "I need HTTPS and clean paths to two internal services", a reverse proxy on one small instance will do. If it is "a deploy takes the site down for ninety seconds", you need health checks, and a proxy in front of a single server will not save you. A third thing gets folded in wrongly: an API gateway, which adds auth, rate limiting and quotas on top, as our guide to AI gateways covers.

Layer 4 vs Layer 7: Types of Load Balancer You Will Meet

Layer 4 works at TCP and UDP. It sees addresses and ports, forwards packets, never parses HTTP, and so is fast, cheap and unable to route on a URL path. AWS Network Load Balancer is the interview example, and what you reach for with gRPC streams, MQTT from IoT devices, or a Postgres read pool.

Layer 7 terminates the connection and routes on method, path, host, header or cookie, which is how /api/* and /static/* reach different target groups from one address. AWS Application Load Balancer is the reference implementation, and its target groups, listener rules and health checks are examined directly in SAA-C03, which 360DT's live AWS Solutions Architect and DevOps course builds and breaks in class rather than describing on a slide. Pick Layer 7 unless you have a reason not to, usually a non HTTP protocol or a need for the client IP to arrive untouched.

Both self hosted options moved recently enough that stale tutorials will mislead you. Nginx shipped 1.30.5 on its stable branch on 15 September 2026, and the 1.30 branch brought cookie based session persistence, Early Hints and Multipath TCP into the open source build, features that used to mean paying for NGINX Plus. HAProxy 3.2 is the current LTS line, with 3.2.25 released on 28 September 2026 and security support to 1 April 2030. For a balancer you will not think about for three years, HAProxy 3.2 LTS is the defensible pick.

Also read: What is Kubernetes? covers where a cluster takes this job over through Services and Ingress.

Where You Will Meet Load Balancing in Real Work

The zero downtime deploy. Where load balancing first earns its keep, and not about traffic volume at all. With one server, a deploy means a gap. With two target groups, you register the new version, wait for health checks, shift the listener and drain the old one, and nobody sees anything. That is as much CI/CD work as networking, so if the deploy half is unfamiliar, start with how a CI/CD pipeline works.

Serving a model behind an endpoint. A model leaving the notebook needs several replicas with a balancer in front, because GPU inference is exactly the uneven workload least connections exists for. Serving, autoscaling and rollout sit inside the AI-300 objectives that 360DT's MLOps Engineer course works through live.

Routing across model providers. A pipeline that calls one hosted model and falls back to a second on a rate limit is load balancing with different words on the boxes: health check becomes error rate, algorithm becomes a cost and latency policy. Students build that layer in 360DT's AI Engineer course alongside their RAG and agent work.

The interview. It shows up in the architecture round of nearly every cloud and DevOps interview in India, usually as "walk me through what happens when a target fails a health check". The certifications overview shows which exams test it formally. On Azure, Application Gateway and Front Door are the AZ-305 equivalents, covered in the Azure Solutions Architect and DevOps course.

What Usually Goes Wrong: Health Checks, Timeouts and Sticky Sessions

Here is the desk warning. Nobody gets paged for choosing round robin over least connections. People get paged because a health check returns 200 while the app is broken, or because one timeout is shorter than the one beside it.

The AWS defaults deserve a second look, because the arithmetic is unkind: a 30 second interval with a healthy threshold of 5 puts about 150 seconds between an instance being ready and the balancer trusting it. During a spike driven autoscaling event, that is a long time to be down a server. Changing a timeout takes one command:

aws elbv2 modify-load-balancer-attributes \
  --load-balancer-arn arn:aws:elasticloadbalancing:ap-south-1:123456789012:loadbalancer/app/dispatch/abc123 \
  --attributes Key=idle_timeout.timeout_seconds,Value=120

Sticky sessions deserve the same scepticism. It is genuinely useful that Nginx open source can now issue a session cookie without a paid licence, and genuinely a sign your app keeps state it should not. Use it to survive two weeks, then move sessions to Redis and take it out.

How to Learn Load Balancing Properly

Reading about it plateaus fast. What builds the skill is running the local setup above, then rebuilding it with a real ALB, two target groups and a deliberately broken health check, so you watch a target leave rotation and return. Do it once and the interview answer writes itself.

To sample the teaching first, the free webinars cover cloud architecture at this level, and a free demo class puts you in a live session for an evening.

Related guides

• What Is Infrastructure as Code? because the balancer you configured by hand should live in Terraform before production.
• SAA-C03 vs DOP-C02 if you want to know which AWS exam tests this material first.
• DevOps engineer salary in India for what the skill set this sits inside currently pays.
• Cloud engineer jobs in Chennai for who actually hires for cloud networking work in one Indian market.
• Linux commands cheat sheet for DevOps for the ss, curl and journalctl work debugging a balancer becomes.

Frequently asked questions

In your position I would run the local Nginx exercise tonight and the broken ALB health check this weekend, in that order. Config syntax is the easy half and you will forget it anyway. What stays with you is having watched a target drop out of rotation and come back, because that is what every interview question and every 2 AM page is really testing. When you want the cloud half taught properly, the AWS Solutions Architect and DevOps course is the direct next step, and a free webinar is the cheaper way to judge the teaching.

About this guide. 360 Digital Transformation is an Anthropic Partner. Other certification bodies, vendors and employers named here are not affiliated with us. Tools and versions change quickly; commands and figures cited were checked on 29 September 2026.