Monitoring & Reporting

Endurance provides several complementary ways to monitor appliance health and performance:

SNMP - for integration with an external network monitoring system.
Metric collectors - for exporting metrics to a Prometheus monitoring stack.
Resource utilization graphs - presented directly in the WebUI System Overview.

Each is described in the sections below.

SNMP Reporting

The appliance supports SNMP v1, v2 and v3. By default, SNMP is disabled. For information on enabling SNMP and configuring the SNMP version(s), community string, location, contact, Service IP and port, see SNMP Configuration.

MIB Files

A number of Management Information Base files (MIBs) are available for use with your monitoring system. The appliance includes all the standard Linux MIBs for general system monitoring, together with a set of custom loadbalancer.org MIBs for monitoring layer 7 services. All MIB files are located in /usr/share/snmp/mibs on the appliance. The custom MIBs can also be downloaded using the links below.

General - For network stats, memory usage, CPU load, disk usage etc., the standard Linux MIBs can be used (for example HOST-RESOURCES-MIB, IF-MIB and UCD-SNMP-MIB).

Layer 7 Services - For monitoring layer 7 (HAProxy) services, the following custom MIBs are provided: LBO-MIB.txt, L7STAT-EXPERIMENTAL-MIB.txt & L7INFO-EXPERIMENTAL-MIB.txt.

Layer 4 services are not exposed through a dedicated SNMP MIB. To monitor the state of layer 4 Virtual Services and their associated Real Servers, use the System Overview (see Using the System Overview) or the API (see Application Programming Interface (API)).

The following sections include examples that demonstrate how SNMP values can be retrieved. Each command can be run at the console, via an SSH session, or using the Local Configuration > Execute shell command WebUI option. The examples use SNMP v2 with a community string of public against the local appliance - adjust the version, community string and target address to match your own configuration.

The appliance does not load the MIB definitions automatically, so by default responses are returned using numeric OIDs. To display the human-readable MIB object names shown in the layer 7 examples below (e.g. l7InfoName), add the -m ALL option to load all installed MIBs, or load a specific MIB with -m <MIB-NAME> (e.g. -m +L7INFO-EXPERIMENTAL-MIB).

Monitoring System Stats using SNMP

General system statistics - such as CPU load, memory usage, disk usage and network throughput - are retrieved using the standard Linux MIBs.

For example, to display the standard system group (system name, uptime, contact, location etc.):

snmpwalk -v 2c -c public localhost 1.3.6.1.2.1.1

SNMPv2-MIB::sysDescr.0 = STRING: Linux lb1 5.14.0-570.58.1.el9_6.x86_64 #1 SMP PREEMPT_DYNAMIC x86_64
SNMPv2-MIB::sysObjectID.0 = OID: NET-SNMP-MIB::netSnmpAgentOIDs.10
DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (10783) 0:01:47.83
SNMPv2-MIB::sysContact.0 = STRING: admin
SNMPv2-MIB::sysName.0 = STRING: lb1
SNMPv2-MIB::sysLocation.0 = STRING: Datacenter
…

The sysContact and sysLocation values shown above are the Contact and Location configured on the SNMP configuration page.

Per-interface network statistics are available under the IF-MIB:

snmpwalk -v 2c -c public localhost IF-MIB::ifTable

Memory, CPU and disk statistics are available under the UCD-SNMP-MIB and HOST-RESOURCES-MIB, for example:

snmpwalk -v 2c -c public localhost HOST-RESOURCES-MIB::hrStorageTable

Monitoring Layer 7 VIPs & RIPs using SNMP

The root OID for layer 7 services is 1.3.6.1.4.1.54849. The examples below use the -m ALL option so that the MIB object names are resolved (see the note above).

A large amount of information is available - the layer 7 MIB files include comprehensive descriptions of all OIDs.

To view general layer 7 (HAProxy) process information (the l7Info group) - this gives similar output to the HAProxy "show info" command:

snmpwalk -v 2c -c public -m ALL localhost 1.3.6.1.4.1.54849.1.1

L7INFO-EXPERIMENTAL-MIB::l7InfoLastPolled.0 = Timeticks: (676842160) 78 days, 8:07:01.60
L7INFO-EXPERIMENTAL-MIB::l7InfoUpdateInterval.0 = INTEGER: 4000
L7INFO-EXPERIMENTAL-MIB::l7InfoLastError.0 = Gauge32: 0
L7INFO-EXPERIMENTAL-MIB::l7InfoName.0 = STRING: HAProxy
L7INFO-EXPERIMENTAL-MIB::l7InfoVersion.0 = STRING: 2.4.22-f8e3218
L7INFO-EXPERIMENTAL-MIB::l7InfoReleaseDate.0 = STRING: 2023/02/14
L7INFO-EXPERIMENTAL-MIB::l7InfoUptime.0 = STRING: 0d 0h00m25s
L7INFO-EXPERIMENTAL-MIB::l7InfoNode.0 = STRING: lb1
…

Accessing VIP (frontend) information

Per-frontend (VIP) statistics are held in the l7Frontend table of the L7STAT-EXPERIMENTAL-MIB. List the configured frontends by their proxy name using the l7FePxname object:

snmpwalk -v 2c -c public -m ALL localhost l7FePxname

The full frontend statistics table can be displayed using snmptable:

snmptable -v 2c -c public -m ALL localhost l7Frontend

Accessing RIP (backend server) information

Per-server (Real Server) statistics are held in the l7Server table, and per-backend statistics in the l7Backend table. List the Real Servers by their service name using the l7SrvSvname object:

snmpwalk -v 2c -c public -m ALL localhost l7SrvSvname

The full backend and per-server statistics tables can be displayed using snmptable:

snmptable -v 2c -c public -m ALL localhost l7Backend
snmptable -v 2c -c public -m ALL localhost l7Server

SNMPv3

When SNMP v3 is configured, include the v3 authentication (and, if configured, privacy) options with each command. For example, for authentication without privacy:

snmpwalk -v3 -u <USM-USERNAME> -l authNoPriv -a <AUTH-ALGORITHM> -A <AUTH-PASSPHRASE> localhost 1.3.6.1.4.1.54849

The USM Username, authorization algorithm and passphrase must match the values configured on the SNMP configuration page (see SNMP Configuration).

Metric Collectors

In addition to SNMP, the appliance can export metrics to an external Prometheus monitoring stack. The built-in Prometheus collector can provide both layer 7 (HAProxy) metrics and local node (system) metrics. For details on enabling and configuring the collector, see Collectors.

Resource Utilization Graphs

The System Overview presents a set of resource utilization graphs that provide an at-a-glance view of appliance health, including:

Network throughput
System load average
Memory usage

These graphs are shown in the lower portion of the System Overview. Clicking the "eye" icon on the CPU, Memory, Disks, Connections or graph widgets opens the System Details view, where a larger, historical graph can be displayed for a selected metric and time period. For more information, see Viewing Detailed Metrics.