#RANDOM SELECTOR BETWEEN TWO OPTIONS PLUS#

Perhaps one traveler has misplaced his or her documentation, or arouses suspicion in the immigration officer: This approach works adequately, until there’s a delay in one of the queues. Round Robin is the default load‑balancing algorithm used by NGINX: Once a traveler is directed to the last queue, the process repeats from queue A. In this approach, the guide selects each queue in rotation – the first traveler is directed to queue A, next traveler to queue B, and so on. Round robin is a naive approach to load balancing. Let’s consider how well some of the possible algorithms work in a distributed load‑balancing scenario like the arrivals hall. The methods available to the guides for selecting the best server correspond to load‑balancing algorithms.The guides maximize efficency by selecting the best server for each request.The immigration desks are (backend) servers, each processing a backlog of requests.You and your fellow travelers are requests, each hoping to be processed as quickly as possible.If we think of the guides as load balancers: Their job is to direct each traveler to join one of the several queues for each immigration desk. Many airports employ guides in the arrivals hall. Photo: Caroline M.A Otieno – Own work, CC BY 2.0 You’ve just landed after a long international flight, and along with 400 other travelers, have walked into a busy arrivals hall. Let’s begin with what might be a familiar situation. In NGINX and NGINX Plus, it’s implemented as a variation of the Random load‑balancing algorithm, so we also refer to it as Random with Two Choices. The algorithm is referred to in the literature as “power of two choices”, because it was first described in Michael Mitzenmacher’s 1996 dissertation, The Power of Two Choices in Randomized Load Balancing. Cluster topologies using distributed load balancersĪ common instance of this scenario occurs when you use NGINX Ingress Controller for Kubernetes, with one load‑balancing instance per Kubernetes node. This scenario is not just observed when you scale out in high‑performance environments it’s also observed in containerized environments where multiple proxies each load balance traffic to the same set of service instances.

The “power of two choices” approach is not as effective on a single load balancer, but it deftly avoids the bad‑case “herd behavior” that can occur when you scale out to a number of independent load balancers. Why Do We Need a New Load‑Balancing Algorithm?Ĭlassic load‑balancing methods such as Least Connections work very well when you operate a single active load balancer which maintains a complete view of the state of the load‑balanced nodes.

#RANDOM SELECTOR BETWEEN TWO OPTIONS PLUS#

New use cases sometimes require new load‑balancing algorithms, and in NGINX Plus R16 and NGINX Open Source 1.15.1 we added a new method that is particularly suitable for distributed load balancers: an implementation of the “power of two choices” algorithm.

Random selector between two options