Load balancing concepts

This chapter covers the background needed to plan a deployment - the load balancing methods and algorithms the appliance provides, the topologies it can be deployed in, and the terminology used throughout the rest of this manual. If you are already familiar with load balancing, you may prefer to go straight to Appliance Deployment.

The Basics

The Loadbalancer.org Endurance appliance enables multiple servers to be combined into a cluster. This enables inbound requests to be distributed across multiple servers which provides improved performance, reliability and resilience. Appliances can also be deployed in a multi-node cluster (our recommended solution) which creates a highly available load balancing solution - for more information, see High availability.

Supported Protocols

Endurance is able to load balance virtually any TCP or UDP based protocol including HTTP, HTTPS, FTP, SMTP, RDP, SIP, IMAP, POP, DNS etc. The protocols that a particular service handles depend on the type of service:

Service type	Protocols
Layer 4	TCP, UDP, or both - selected using the Layer 4 mode field of the frontend. For more information, see Layer 4 Services.
Layer 7	HTTP mode for HTTP/1.1 and HTTP/2 traffic, or TCP mode for all other TCP based protocols such as HTTPS, RPC, RDP, FTP and SIP - selected using the Layer 7 mode field of the frontend. For more information, see Layer 7 Services.
GSLB	DNS. The service answers DNS queries for the delegated sub domain(s); the health checks that determine which answers are returned support TCP, HTTP(S) and custom external checks. For more information, see GSLB Services.

Service type

Protocols

Layer 4

TCP, UDP, or both - selected using the Layer 4 mode field of the frontend. For more information, see Layer 4 Services.

Layer 7

HTTP mode for HTTP/1.1 and HTTP/2 traffic, or TCP mode for all other TCP based protocols such as HTTPS, RPC, RDP, FTP and SIP - selected using the Layer 7 mode field of the frontend. For more information, see Layer 7 Services.

GSLB

DNS. The service answers DNS queries for the delegated sub domain(s); the health checks that determine which answers are returned support TCP, HTTP(S) and custom external checks. For more information, see GSLB Services.

Layer 4 & Layer 7

Load balancing at layer 4 and layer 7 is supported. LVS (Linux Virtual Server) is used at layer 4, HAProxy is used at layer 7 and Polaris is used for GSLB. All three service types are configured from the same Services page using template based wizards, and a single appliance can run any combination of them at the same time. For guidance on choosing between layer 4 and layer 7, see Layer 4 vs Layer 7.

What is a Service?

A service is the key building block of any load balancing configuration - it provides the endpoint that clients connect to. Each service is made up of the following components:

Component	Description
Frontend	Defines the Floating IP address, mode and port(s) that the service listens on.
Backend	Defines the pool of load balanced (Real) servers that requests are distributed to, together with the balance mode, health check, persistence and other settings that apply to that pool.
ACL rule	Connects the frontend to a backend. A Default ACL rule is created automatically when the service is added, and sends all traffic on the frontend to the backend defined in the wizard.

Component

Description

Frontend

Defines the Floating IP address, mode and port(s) that the service listens on.

Backend

Defines the pool of load balanced (Real) servers that requests are distributed to, together with the balance mode, health check, persistence and other settings that apply to that pool.

ACL rule

Connects the frontend to a backend. A Default ACL rule is created automatically when the service is added, and sends all traffic on the frontend to the backend defined in the wizard.

A service can have more than one frontend and more than one backend. Where a Layer 7 service has multiple backends, additional ACL rules are used to steer each request to the appropriate backend based on the content of the request. A Layer 4 frontend only ever has the single default rule. For more information, see ACL Rules.

What is a Floating IP?

A Floating IP is the address that clients connect to. Floating IPs are active on one node at a time and can move or "float" between nodes to enable HA failover - they are controlled by Pulse to ensure that only the active node owns the address at any time. It’s therefore not possible to configure a Floating IP on the same address as an interface.

Floating IPs are also used to provide a "floating default gateway" for configurations that require the load balancer to be the default gateway for the Real Servers, such as Layer 4 DNAT mode and Layer 7 proxy mode with TProxy enabled.

A Floating IP can be created as part of the service wizard, or managed independently under Network > IPs > Floating IPs. For more information, see Floating IPs.

Floating IPs are one of three types of IP address used by the appliance. Static IPs are used by the appliance itself - the WebUI, API, SSH and PulseHA - and Service IPs associate a name with a Static IP so that it can be used by appliance services such as SNMP and the Prometheus collector. For more information, see IPs.

Load Balancing Algorithms

The appliance supports several different load balancing algorithms, referred to in the WebUI as the Balance mode. Each has its own advantages and disadvantages and it depends on the specific application which is the most appropriate to use. The balance mode is configured on the backend of a service, and the options available depend on the type of service. Usually the default method - Weighted Least Connection at layer 4 and Weighted Least Connected at layer 7 - is a good solution which works well in most situations.

The weighted methods take each Real Server’s Weight into account. Weightings are relative, so it makes no difference if Real Server #1 and #2 have weightings of 50 and 100 respectively or 5 and 10. A server with a weight of 200 will receive 4 times the number of requests than a server with a weight of 50. For Layer 4 services the weight can be set between 0 and 100, and for Layer 7 services between 0 and 256. The default is 100 in both cases, and a weight of 0 places the server in drain mode. For GSLB services the endpoint weight can be set between 0 and 10, and a weight of 0 disables the endpoint.

Layer 4 Balance Modes

Balance mode	Description
Weighted Least Connection	Requests are distributed to the Real Servers with the fewest connections, relative to each Real Server’s weight. Servers with a higher weight receive more requests. This is the default option.
Weighted Round Robin	Requests are distributed to the Real Servers in a sequential manner, relative to each Real Server’s weight. Servers with a higher weight receive more requests, and servers with equal weights receive an equal distribution of new requests.
Source Hash	Requests are distributed to the Real Servers by looking up the client’s source IP address in a statically assigned hash table. Because the same client address always hashes to the same Real Server, this provides a simple form of persistence.

Balance mode

Description

Weighted Least Connection

Requests are distributed to the Real Servers with the fewest connections, relative to each Real Server’s weight. Servers with a higher weight receive more requests. This is the default option.

Weighted Round Robin

Requests are distributed to the Real Servers in a sequential manner, relative to each Real Server’s weight. Servers with a higher weight receive more requests, and servers with equal weights receive an equal distribution of new requests.

Source Hash

Requests are distributed to the Real Servers by looking up the client’s source IP address in a statically assigned hash table. Because the same client address always hashes to the same Real Server, this provides a simple form of persistence.

Layer 7 Balance Modes

Balance mode	Description
Weighted Least Connected	New connections are sent to the Real Server with the fewest active connections, relative to each Real Server’s weight. This is the default and recommended option.
Weighted Round Robin	Connections are distributed across the Real Servers in turn, relative to each Real Server’s weight.
First	The first Real Server with available connection slots receives the request. Servers are chosen from the lowest numeric identifier to the highest, which defaults to the server’s position in the backend. Once a server reaches its connection limit, the next server is used.
Source	The Real Server is selected based on a hash of the client source IP address, providing a simple form of persistence.

Balance mode

Description

Weighted Least Connected

New connections are sent to the Real Server with the fewest active connections, relative to each Real Server’s weight. This is the default and recommended option.

Weighted Round Robin

Connections are distributed across the Real Servers in turn, relative to each Real Server’s weight.

First

The first Real Server with available connection slots receives the request. Servers are chosen from the lowest numeric identifier to the highest, which defaults to the server’s position in the backend. Once a server reaches its connection limit, the next server is used.

Source

The Real Server is selected based on a hash of the client source IP address, providing a simple form of persistence.

GSLB Balance Modes

Balance mode	Description
Weighted round robin	Round robin with weighting. This is the default option.
Topology weighted round robin	As above, but the topology (region) information is also considered, to direct clients to endpoints in the same region / data center.
Failover Group	The first healthy endpoint is handed out continuously unless it becomes unhealthy, in which case the next healthy endpoint is used, and so on. Can be used for active-backup scenarios.

Balance mode

Description

Weighted round robin

Round robin with weighting. This is the default option.

Topology weighted round robin

As above, but the topology (region) information is also considered, to direct clients to endpoints in the same region / data center.

Failover Group

The first healthy endpoint is handed out continuously unless it becomes unhealthy, in which case the next healthy endpoint is used, and so on. Can be used for active-backup scenarios.

Real Server Feedback Agent

To complement the methods above, Loadbalancer.org appliances also support Real Server agents. This permits the load balancing algorithm to be dynamically modified based on each Real Server’s running characteristics. For example, one Real Server could have a run-away process that is consuming excessive CPU resources or RAM. Without the agent, the load balancer has no way of knowing this and would continue to send requests to the overloaded server based on the algorithm selected. With the agent installed on the Real Server, feedback is provided to the load balancer and the algorithm is then adjusted to reduce requests that are sent to that server.

For Layer 7 services, the agent is enabled per backend using the Enable option on the Backend Configuration - Layer 7 advanced wizard step (available in Advanced Mode), where the Agent port (default 3333) and the Feedback agent interval are also configured - for more information, see Backend Configuration - Layer 7 advanced. For GSLB services, the "Feedback_Agent_Weight" health check template can be used to create a dynamic weight health check script that interacts with the agent - for more information, see GSLB Healthcheck Scripts.

For details of installing and configuring the agent on the Real Servers, see Server Feedback Agent.

Layer 4 vs Layer 7

A fundamental choice when setting up the load balancer is whether to configure the services at layer 4 or layer 7.

How They Differ

At layer 4 the primary protocols used are TCP and UDP. These protocols are not aware of upper level protocols such as FTP, HTTP, HTTPS, DNS, RDP etc. Therefore the load balancer can only make load balancing decisions based on details available at layers 4 and below such as port numbers and IP addresses. At layer 7, the load balancer has more information to make load balancing related decisions since more information about upper level protocols is available.

Layer 7 load balancing uses a proxy at the application layer (HAProxy). Requests are terminated on the load balancer, and the proxy generates a new request which is passed to the chosen Real Server.

Performance

Due to the increased amount of information at layer 7 and the fact that a proxy is in use, performance is not as fast as at layer 4. If raw throughput is a primary concern, then layer 4 is probably the better choice.

Persistence

Persistence (aka affinity or sticky connections) is the ability to ensure that a specific client connects back to the same server within a specific time period. It’s normally required when the session state is stored locally on the Real Server rather than in a separate database. At layer 4, source IP persistence is the only option. At layer 7, additional methods are available such as HTTP cookie persistence where the load balancer sets a cookie to identify the session, and application session persistence where an existing session parameter set by the web application on the Real Servers is used. For the full list, see Default Persistence Options.

Real Server Changes

For layer 4 DR mode, the "ARP Problem" must be solved - for more information, see DR Mode Considerations. For layer 4 DNAT mode, the default gateway on each Real Server must be the load balancer. For layer 4 SNAT mode and layer 7 proxy mode the Real Servers do not need to be changed in any way, unless TProxy is used at layer 7 in which case return traffic must be routed back via the load balancer.

Transparency

Transparency refers to the ability to see the originating IP address of the client. For layer 4 DR mode and DNAT mode, connections are transparent. For layer 4 SNAT mode and layer 7 proxy mode, the IP address of the load balancer is recorded as the source address - for layer 7 proxy mode this can also be set to another address owned by the load balancer using the Backend source address field. For layer 7 proxy mode, additional configuration steps can be taken to force the client IP to be logged. Options include using TProxy to re-write the source address, or enabling support for X-Forwarded-For or Proxy Protocol headers. For more information, see Transparency at Layer 7.

Other Considerations

Does the Application Cluster Correctly Handle its own State?

Load balancers work most effectively if the application servers are completely stateless. This means that if an application server (i.e. Real Server) fails and is automatically taken out of the cluster, then all current user sessions will be transferred to other servers in the cluster without users needing to re-authenticate to the application.

Web based applications are inherently stateless and are an ideal candidate for load balancing. However, do your web servers store files and other information on local drives?

Images (jpeg, png, gif etc.)
Files (html, php, asp etc.)

If so, these files need to be on shared storage or they need to be replicated to all nodes in the cluster.

Replication Solutions for Shared Data

On Linux systems, Rsync can be used to replicate files. On Windows systems Rsync can also be used, although the Robocopy command may be preferred. Typically, content is uploaded to one primary server and then replicated to the other servers in the cluster.

Solutions for Session Data

Standard ASP and PHP session data is stored locally by default, leaving your session data in a local store will prevent you from implementing seamless application server fail-over in your cluster. If an application server fails, all the local session data will be lost and users will need to re-authenticate and possibly lose shopping baskets etc.

This problem can be resolved by implementing a shared persistent data store for the cluster. This is usually achieved using a shared backend database.

What if the Application is not Stateless?

Some protocols require state to be maintained such as:

SSH
FTP
SMTP

Also, it may not be an option to modify HTTP/HTTPS based applications to handle shared session data.

For these cases persistence can be used. Transparent fail-over will no longer be possible, but increased capacity and manageability are still worthwhile benefits of implementing a load balancer.

Default Persistence Options

For layer 4, persistence is based on the client source IP address and is enabled by default for all new services. The default timeout is 5 minutes and is reset on every new client connection, so a 5 minute setting could last for hours if the client is active and regularly refreshes their connection. The setting is configured on the Backend Configuration - Persistence wizard step, which is available in Advanced Mode.

For layer 7, persistence is configured on the Backend Configuration - Persistence wizard step and the following Mode options are supported:

Mode	Description
None	No persistence; each request is load balanced independently.
Source IP	Clients are pinned to a Real Server based on their source IP address. This is the default option.
App Session	Persistence based on an application session parameter.
HTTP Cookie	Persistence is maintained using an HTTP cookie inserted by the load balancer.
HTTP Cookie and Source IP	Combines HTTP Cookie persistence with Source IP as a fallback.
X-Forwarded-For	Persistence based on the client address in the X-Forwarded-For header.

Mode

Description

None

No persistence; each request is load balanced independently.

Source IP

Clients are pinned to a Real Server based on their source IP address. This is the default option.

App Session

Persistence based on an application session parameter.

HTTP Cookie

Persistence is maintained using an HTTP cookie inserted by the load balancer.

HTTP Cookie and Source IP

Combines HTTP Cookie persistence with Source IP as a fallback.

X-Forwarded-For

Persistence based on the client address in the X-Forwarded-For header.

Cookie based persistence requires the Layer 7 service to be configured in HTTP mode. For more information on the layer 7 persistence settings, including the persistence Timeout and stick table options, see Backend Configuration - Persistence.

Objectives & Outcomes

It’s important to have a clear understanding of the objectives and required outcomes to ensure a successful load balancer implementation. Are you looking for increased performance, reliability, ease of maintenance or all three?

Performance

A load balancer can increase performance by allowing multiple commodity servers to handle the workload of one application.

Availability

Running an application on one server introduces a single point of failure. Utilizing a load balancer to present multiple servers improves application availability but moves the point of failure to the load balancer. To mitigate this, multiple nodes should be deployed. For more information, see High availability.

Maintenance

Using the appliance, servers can be brought on and off line to perform maintenance tasks, without disrupting users.

In order to achieve all three objectives, the application must handle persistence correctly. For more information, see Does the Application Cluster Correctly Handle its own State?.

Loadbalancer.org Terminology

Load Balancer
ADC

An IP based traffic manager for server clusters.

Node

An individual Endurance appliance. Up to 4 nodes can be clustered to provide high availability. See High availability.

Service
Virtual Service

The main building block used to configure load balanced services. It defines the IP address clients connect to, which Real Servers are load balanced and other settings such as health check options, persistence options and timeout settings.

Frontend

The part of a Service that defines the Floating IP address, mode and port(s) that clients connect to.

Backend

The part of a Service that defines the pool of Real Servers that requests are distributed to, together with the balance mode, health check and persistence settings that apply to that pool.

ACL Rule

Access Control List rule. Links a frontend to a backend and, for Layer 7 Services, steers requests to different backends based on the content of the request. See ACL Rules.

Real Server

The actual backend server being load balanced. Multiple Real Servers are associated with a backend.

Health check

A test that determines whether a Real Server is available. Health checks are defined centrally and selected when configuring a backend. See Healthchecks.

Floating IP

The IP address associated with a Service, the address presented to connecting clients. Enables services to float (move) between the nodes in the cluster. See Floating IPs.

Static IP

An IP address used by the appliance itself - the WebUI, API, SSH and PulseHA. See Static IPs.

Service IP

A name associated with a Static IP so that it can be used by appliance services such as SNMP and the Prometheus collector. See Service IPs.

IP Group
Subnet

A subnet defined on the appliance, associated with an interface on each node. Static IPs and Floating IPs are allocated from an IP group. See Subnets.

Region

A named group of one or more subnets, used by the GSLB Topology weighted round robin balance mode to direct clients to endpoints in the same location. See Regions.

VIP

Virtual IP address - alternative name for the IP address associated with a Service. Also used as an acronym for a Service / Virtual Service.

RIP

Real IP address - the IP address of a backend server in the cluster. Also used as an acronym for a Real Server.

GSLB

Global Server Load Balancing - DNS based load balancing that distributes clients across multiple sites or data centers. See GSLB Services.

PulseHA

The clustering service that manages node membership, failover and ownership of the Floating IPs. See High availability.

WebUI

Web User Interface. Used to configure and manage the appliance.

Layer 4

Part of the seven layer OSI model. Also a descriptive term for load balancing methods that routes packets based on TCP/IP header information.

Layer 7

Part of the seven layer OSI model. Also a descriptive term for a proxy based load balancing method that distributes packets based on the entire TCP/IP header and also the payload information at the application layer.

DR Mode

Direct Routing (aka DSR/Direct Server Return) is a standard layer 4 load balancing technique that distributes packets by altering only the destination MAC address of the packet.

DNAT Mode

Destination Network Address Translation is a standard layer 4 load balancing technique that changes the destination of packets to and from the VIP (external subnet to internal cluster subnet).

Layer 4 SNAT Mode

Source Network Address Translation - similar to DNAT mode but also modifies the source address of all outgoing traffic to be the load balancer.

Layer 7 Proxy Mode
Layer 7 SNAT Mode

Load balancing method that uses a proxy (HAProxy) for all incoming & outgoing traffic.

TProxy

Transparent proxy. Used at layer 7 to re-write the source address of each packet so that the Real Servers see the original client IP address. See Using TProxy to Modify the Source IP Address.

SSL Passthrough

The SSL certificate is installed on each Real Server. Data is encrypted from the client all the way through to the Real Server, providing full end-to-end encryption.

SSL Termination
SSL Offloading

The SSL certificate is installed on the load balancer. Data is encrypted from the client to the load balancer, then passed to the Real Servers unencrypted.

SSL Bridging

SSL certificates are installed on the load balancer and the Real Servers. Data is encrypted from the client to the load balancer and re-encrypted from the load balancer to the Real Servers. This provides end-to-end encryption while still allowing the load balancer to inspect/modify the traffic.

One-Arm

The load balancer has one physical network card connected to one subnet.

Two-Arm

The load balancer has two interfaces connected to two subnets - this can be achieved by using two network adapters, or by creating VLANs on a single adapter.

The Default Gateway for a backend server in the cluster.

Interface name

The name assigned to a network interface by the operating system, e.g. enp1s0, enp2s0. Each interface can also be given a friendly Name in the WebUI, e.g. Network-1. See Interfaces.