What is a load balancer and how does it work?
A load balancer distributes incoming network traffic across multiple servers so that no single server is overwhelmed, which keeps an application available, responsive and able to scale. It sits in front of a group of servers, presents one address to the world, and quietly decides — request by request — which healthy server should do the work. That simple job turns a collection of individual machines into something that behaves like one reliable system: servers can fail, be replaced or be added without users noticing. Every major cloud provides load balancing as a managed service, and the concept sits underneath high availability, auto-scaling and most of the architecture questions on associate-level certification exams. This article explains the problem it solves, how it makes its decisions, and what the AWS and Azure versions are called.
The problem: one server is a bottleneck and a single point of failure
Run an application on one server and you have accepted two risks at once. The first is capacity: a single machine can only handle so many simultaneous users, and once demand passes that ceiling, response times stretch and requests start failing — precisely when the application is most popular. The second is fragility: if that one server crashes, is rebooted for patching, or loses its network, the application is simply gone. One machine is both your bottleneck and your single point of failure.
The obvious fix is more servers — but more servers create a new problem. Users connect to an address, not a fleet; something has to stand in front, share the incoming work fairly, and stop sending traffic to any machine that has died. That something is the load balancer. It is the piece that converts “several servers” into “one dependable application”, and it is why the pattern appears in virtually every production architecture diagram you will ever see.
How it works: health checks and routing decisions
A load balancer’s intelligence comes down to two continuous activities. The first is health checking: it regularly probes every server behind it — requesting a known page, or simply testing the connection — and any server that stops responding correctly is marked unhealthy and removed from the rotation. When the server recovers and passes its checks again, it is quietly restored. Users never see the failure; their requests simply go elsewhere.
The second activity is choosing where each request goes. The simplest algorithm is round-robin — each server takes its turn — while smarter variants send work to the server with the fewest active connections, or weight some servers more heavily than others. One refinement worth knowing is sticky sessions (session affinity): if an application keeps user state in a server’s memory, the load balancer can pin each user to the same server for the duration of their session. Stickiness is a pragmatic crutch rather than a virtue — modern designs prefer storing session state outside the servers so any server can handle any request — and exams like you to know both the mechanism and the trade-off.
Layer 4 vs layer 7, without the jargon
Load balancers come in two broad kinds, named after the network layers they operate at. A layer 4 load balancer works at the transport level: it sees network connections — addresses and ports — and forwards them without opening them up. Because it does so little inspection, it is extremely fast, and it can balance any TCP or UDP traffic, not just web requests.
A layer 7 load balancer understands the application protocol itself — for web traffic, HTTP. Because it can read each request, it can make richer decisions: send requests for one path to one group of servers and another path to a different group, route by hostname, terminate TLS encryption, or redirect traffic. The rule of thumb: layer 7 when you want intelligent routing of web traffic, layer 4 when you want raw speed or are balancing something other than HTTP. Certification scenarios often reduce to spotting which of the two a described workload needs.
The partnership with high availability and auto-scaling
Load balancing is one half of the cloud’s high-availability story. Spread your servers across multiple availability zones, put a load balancer in front, and the failure of an entire data centre becomes a non-event: health checks detect the lost servers and traffic flows to the surviving zone. This zone-spanning pattern is the standard answer to “how do I make this resilient?” in both AWS and Azure architecture.
The other half is auto-scaling, and the two are designed to work as a pair. Auto-scaling adds servers when demand rises and removes them when it falls; the load balancer is what makes that seamless, because new servers register with it and start receiving traffic automatically, while removed servers are drained gracefully. Without a load balancer, auto-scaling would produce servers nobody’s traffic could reach. Our separate explainer on auto-scaling covers that half of the pairing in the same depth.
Load balancing on AWS and Azure
Both major clouds offer managed load balancing, split along exactly the layer 4/layer 7 line described above:
- AWS Application Load Balancer (ALB) — the layer 7 option under the Elastic Load Balancing family: HTTP-aware, with path- and host-based routing, ideal for web applications and microservices.
- AWS Network Load Balancer (NLB) — the layer 4 option: very high performance TCP/UDP balancing with support for static addresses.
- Azure Load Balancer — Azure’s layer 4 service, distributing TCP and UDP traffic across virtual machines.
- Azure Application Gateway — Azure’s layer 7 service: HTTP-aware routing with an optional web application firewall built in.
Where load balancers appear in certification study
Load balancing is core associate-level material on both tracks. The AWS Solutions Architect Associate tests it constantly: choosing ALB versus NLB for a described workload, pairing load balancers with auto-scaling groups across availability zones, and understanding health checks and session handling. The CloudOps Engineer Associate approaches the same services operationally — deploying them, reading their health-check behaviour and troubleshooting why traffic is not reaching targets. On the Azure side, AZ-104 expects you to know when Azure Load Balancer is the answer and when Application Gateway is, and how each fits into a highly available design.
The concept itself is simple enough to hold in one sentence — spread the traffic, skip the dead servers — but the exams reward knowing the consequences: which layer, which product, and how the pieces combine into an architecture that survives failure. For the exam-depth detail, the high-availability and networking lessons in our revision sections pick up where this explainer stops.
Original practice questions, timed mock exams and revision notes. No card, nothing to pay.