Written by Anonymous
on 

Puzzle Piece: Load Balancer

As you do more of these problems, recurring patterns emerge; how to shard NoSQL data, what kinds of sites fit in a single RDS box, etc. This series will try to call out those patterns and pieces specifically, starting from low-level parts. Today we cover the humble load balancer.

TLDR

Use an AWS ALB in multi-AZ and you have infinite requests/second scalability and no Single Point of Failure. If you expect surgy traffic, ex a zonal failover, more work needed. Load balancers are almost never the problem.

Options for load balancing:

  1. Register multiple IPs on the same DNS record and round-robin it.
  2. Delegate to subdomains on different IPs, ex for geographic spread.
  3. Have server-side load balancing set up.

What is Server-Side Load Balancing?

A reverse proxy sits in between traffic coming from the broader internet and your machine.

graph TD A[Client] -->|Sends Request| B(Internet) B --> C{Load Balancer} C -->D[App Server 1] C -->E[App Server 2] C -->F[App Server 3]

The load balancer commonly provides termination services, ex for SSL, or compression/decompression (ex with gzip). Rather than serving HTTP requests itself, a SSLB farms out requests to a pool of app servers that do the actual work. Then, those servers send the HTTP response back through the LB, which sends back to the client. The request originator might get stashed in a new HTTP header sent to the backend, ex HTTP-FORWARDED-FOR.

In-Market Options

AWS

AWS provides 2 real options for load balancing (and one legacy option not to be used, generally). They are:

Application Load Balancer

An ALB has scaling built in, so it scales to handling as many requests as you need. However, it takes time to scale the ALB, so you need to pre-warm it if you expect a spike in traffic. Operates on OSI 7. Announcement in 2016.

Network Load Balancer

An OSI 4 load balancer, an NLB can scale to “millions of requests per second”. Offers higher performance at the expense of required configuration/existing lower in OSI. Announcement in 2017.

Classic Load Balancer

Only recommended for EC2 Classic instances. Operates on both levels 4 and 7. Don’t use this.

Materials

Apache and NGINX

Apache was started in 1995, and NGINX in 2004. Wikipedia. Both are software that does load balancing, ie you must be arranging the machines yourself, unlike AWS solutions.

Metrics + SDI Concerns

AWS ALB: infinite requests per second. Not a single point of failure by itself, if you have presence in multiple AZs. (Otherwise, 1 AZ going down, ex us-east-1, means you are down.)

NLB: Millions of requests per second. Multi-AZ=> not SPoF.

NGINX: 400k-500K requests/second. See here for HA config to avoid SPoF.

Apache: Still looking for metrics.