Introduction

High Availability (HA) for Server Load Balancing (SLB) consists of three ServerIron (SI) services:

Direct Server Return (DSR) enables the return traffic to not be processed by the SI. Instead, the real server directly sends the return traffic to the client. You can apply DSR to each of the HA scenarios (Hot Standby SLB, symmetric SLB, and Sym-Active SLB). For more information, see "DSR".

Hot Standby SLB

Introduction

Since Hot Standby SLB is an HA feature, there is a need for two SIs in the network. If only one device is present and the Hot Standby feature is enabled, the SI will function in "Single-box" mode untill the second SI comes up.

There are two flavors of Hot Standby SLB:

For Hot Standby SLB, one SI is always active while the other SI is always standby (idle).

Hot Standby allows you to configure two SIs to serve as a redundant pair (primary and secondary). If the active SI fails, the idle standby SI assumes the active duties and becomes the new active device.

Hot Standby is the only HA service counting the number of available router-ports and server ports for failover behavior. The SI with the highest number of active ports is declared the active device. In addition to port-count loss, a system reload or crash triggers a failover.

Hot Standby Protocol Operations

When SI-A comes up in a Hot Standby configuration, it comes up in Standby state. By sending Hello messages and observing no other SI is responding to those Hello messages, SI-A assumes Active State. When SI-B comes up, it also goes through Hello-message processing. When SI-B sends Hello messages, SI-A responds to SI-B with SI-A's Active Status. SI-B assumes Standby status. SI-A being in Active State sends these 4 stages of synchronization:

Once the entire synch process is complete, SI-B calculates to see if SI-A has a higher router-port + server-port count or if SI-B itself has the highest count. If SI-A and SI-B tie, then SI-B continues to stay in Standby state. If SI-A has a lesser count of router-port + server-port, SI-B forces SI-A to go to Standby State, and SI-B assumes Active state. From now on, SI-A will be in Active State and SI-B will be Standby State until some event forces a change in their status. Events leading to change in state are as follows:

While the standby SI-B is idle, it is continuously listening to the Active SI-A for fail-over preparation. Active SI-A synchronizes its session table to exactly the same on Standby SI-B. This action takes place the very moment a session is created on Active SI-A. Synchronization of a session involves sesssion creation, session deletion, and age updates. No CLI commands are required to invoke session synchronization from Active SI-A to Standby SI-B. Both SI-A and SI-B perform L2/L3/L4/L7 healthchecks independently. To avoid a loop between SIs, SI-B is converted into a dumb device in Standby State. Therefore, Standby SI-B simply receives session-sync messages from Active, performs Healthchecks, and processes Hello messages. SI-B is completely isolated and does not process any SLB traffic. When SI-A fails, SI-B becomes active and immediately takes over the processing of SLB traffic. Since the sessions are already synchronized from SI-A when it was in Active State, failover is transparent to users.

Despite the solution's stability, having an inactive device (SI-B) with all its VIPs in standby state has been viewed as a limitation by some engineers. Therefore, Foundry created a new HA feature called symmetric SLB (see page Click Here).

Example 1: Standard Hot Standby

Follow these steps to enable the minimum required configuration shown in Figure 8.2. Client connections and server connections must be on the same interfaces on both SIs.

Step 1—On SI-A, place the untagged hot standby port (sync-link) in its own port-specific VLAN and disable STP:

!
vlan 2 by port
 untagged ethe 2/1
 no spanning-tree
!

Placing the hot standby port in its own VLAN prevents unneccessary traffic from going over the directly connected backup link. With Hot Standby, you cannot have spanning-tree configured anywhere on any link connected between the two SIs. See show spanning-tree for more information. By default, spanning tree is usually enabled on switches but disabled on routers.

Step 2—To avoid system conflicts, globally disable spanning-tree on VLAN 1:

!
vlan 1 name DEFAULT-VLAN by port
 no spanning-tree
!

Step 3—Configure the server backup port, shared chassis-MAC address between the SIs, and any connected router-ports:

The server backup ethernet command must be configured exactly the same on both SIs. It has three parameters:

<portnum>—Specifies where the syn-link is connected, which is the port that connects this SI switch to the other one. In the example, 2/1 is the port number.

<mac-addr>—Specifies the chassis-MAC address of one of the SIs. On XL systems, this MAC is the MAC of port 1. Use show int brief to display the MACs of all the ports.

<vlan-id>—Specifies a VLAN you want to use for active-standby synchronization traffic. In this example, the sync-link Hot Standby port is in VLAN 2.

The server router-ports command enables the SI to count the number of upstream (or downstream) router ports connected to the switch. Both SIs must use the same router-ports numbers, such as 2/3 in this example. The reason is the standby SI is a dummy device that learns nothing, such as MACs, on its own.

Step 4—Save the configuration:

SLB-SI-A#wr mem
.Write startup-config in progress.
.Write startup-config done.
SLB-SI-A# reload

Tip: Be sure to reload the software after configuring or changing the server backup port number. If you change the port number of the backup while the SI is load balancing, clients will not be able to ping the VIP.

Step 5—Configure the second SI (SI-B). On this system, port 2/1 is the hot standby port. Using the same port numbers and MAC address is a requirement. Notice the chassis-MAC address on each SI matches:

!
server backup ethe 2/1 00e0.5201.0c72 vlan-id 2
server router-ports ethernet 2/3
server router-ports ethernet 2/4
!
vlan 1 name DEFAULT-VLAN by port
 no spanning-tree
!
vlan 2 by port
 untagged ethe 2/1
 no spanning-tree
!

SLB-SI-B#wr mem
.Write startup-config in progress.
.Write startup-config done.
SLB-SI-B#reload

Step 5—Use show server backup and show log to obtain a clear picture of the SI's status in the Hot Standby configuration:

Following is a series of screen shots displaying the different stages of reload and how an SI comes up in a Hot Standby configuration.

SI-A is running in single-box mode, since SI-B is not yet discovered.

Now SI-B comes up. SI-A is already up and running.

.

Example 2: VIP and Servers in Different Subnets

Example 3: Source-NAT in Hot Standby

The server source-nat command is added to the following configuration on both SIs. However, seamless failover cannot be achieved here. See "Example 4: Seamless Failover in Hot Standby When Source-NAT Enabled".

Example 4: Seamless Failover in Hot Standby When Source-NAT Enabled

Failover Options

Starting in software release 07.3.03, Hot Standby is enhanced to cause failover even if the active and standby SIs have the same number of healthy router ports, when the active SI has fewer healthy server ports. In previous releases, failover in this situation occurs only if the active SI has zero healthy server ports.

Here is the algorithm for hot standby in software release 07.3.03:

The algorithm is not configurable. A configuration saved using an earlier release will use the new algorithm once you upgrade the software.

backup-group

You can configure up to eight hot-standby pairs within a single L2 broadcast domain. To enable this support, use server backup-group to configure a backup group ID on each of the SIs, so that both SIs in a given pair have the same ID. The backup group ID uniquely identifies the pair.

When you configure a backup group ID, both SIs in a hot-standby pair use the ID when exchanging backup information. If a SI receives a backup information packet but the packet's backup group ID does not match the SI's backup group ID, the SI discards the packet.

If the broadcast domain contains multiple hot-standby pairs, you must configure backup group IDs on all pairs. If the broadcast domain contains only one hot-standby pair, you do not need to configure a backup group ID.

[no] server backup-group <id>

The <id> parameter specifies the backup group ID and can be a number from 1 - 7. The default value is 0. Enter the same ID on both SIs in a hot-standby pair. Do not enter the same ID on a SI that is not one of the SIs in the hot-standby pair.

ServerIron(config)#server backup-group 1

backup-timer

The standby SI assumes the active role if the standby SI does not receive a Hello message or Layer 4 session synchronization data from the active SI within a certain number of seconds since receiving the last Hello message or synchronization data.

By default, the standby SI waits one second since receiving the last Hello message or data to receive a new message or data. If the standby SI does not receive a new Hello message or data within one second, the standby SI assumes that the active SI is no longer available and takes over the active role.

In some configurations, particularly configurations in which the active SI is performing a lot of processing, it is possible for frequent failovers to occur. In this situation, although the active SI is still available and actively serving load balancing or other requests, the active SI does not always send the Hello message or synchronization data in time for the standby SI. As as result, the standby SI takes over the active role. If similar conditions cause the newly active SI to sometimes miss sending the Hello messages or synchronization data in time, failover occurs again.

You can prevent unnecessary state flapping between the two SIs by increasing the backup timer. When you increase the backup timer, the standby SI waits longer to receive new Hello messages or synchronization data from the active SI. As a result, flapping is reduced or eliminated.

To change the backup timer, use one of the following methods.

Use server backup-timer to change the backup timer on a SI in an active-standby pair.

[no] server backup-timer <time>

The <time> parameter specifies how long this SI, when it is the backup SI, will wait for a Hello message or synchronization data from the active SI before assuming the active SI is no longer available. You can specify a value from 5 (one half second) - 100 (10 seconds), in units of 100 milliseconds each. The default is 10 (one second).

SI(config)#server backup-timer 50

This command sets the backup timer to 5 seconds (50 * 100 milliseconds).

backup-preference

You can configure one of the SIs in the active-standby pair to always be the active SI. When you enable server backup-preference on one of the SIs, that SI is always the active one by default. The only event that can cause the other SI to be the active one is unavailability of the default active SI or its link to the backup SI. To allow graceful insertion, the SI does not immediately assume the active role, but instead waits for a configurable number of minutes before taking the active role.

[no] server backup-preference <wait-time>

The <wait-time> parameter specifies how long the SI waits before assuming the active role. The SI does not immediately become the active SI but instead waits the number of minutes you specify. You can specify from 5 - 30 minutes. This parameter does not have a default.

SI(config)#server backup-preference 5

fwd-l4-sync

In a hot-standby configuration, the active SI and the backup SI continuously communicate synching messages. These synching messages contain Layer 4 - 7 session status information and are only used by the SIs.

Some of the messages may travel over a non-dedicated private link between the two SIs. Another SI may be in the middle of this link, acting as a Layer 2 or Layer 3 Switch passing traffic between the active and backup SIs.

In this situation, messages sent between the active and backup SIs may be intercepted and dropped by the SI in the middle, and not forwarded to the active or backup SIs. This could cause loss of synch between the active and backup SIs. To prevent this from happening, use [no] server fwd-l4-sync to configure the SI in the middle to simply forward the synching messages and not intercept them.

Enter the following command on the SI connecting the active and the backup SIs:

ServerIron(config)#server fwd-l4-sync

Real/Virtual Server Configuration Example

Suppose you want to configure a second switch, SI2, to serve as the backup or standby switch for SI1. Each switch will be configured with the same SLB configuration, supporting the following TCP/UDP ports: HTTP, SSL, FTP, and Telnet.

The private link, which provides the connection between the active and standby switches, will be configured as a trunk group with ports 13 and 14 as members.

ServerIron# config term
ServerIron(config)# trunk server ethernet 13 to 14

This note refers only to HA links and applies even if the other device is an SI. On stackable XL devices, you must use the trunk server parameter instead of the default trunk switch parameter, regardless of the device type you are connecting to. The one exception to this rule is when a stackable SI is load balancing Layer 2 firewalls. In this case, use the switch parameter. Use the server parameter in all other cases. Both options are generally supported, depending on your application.

For non-HA trunk links connected to other Layer 2 switches, use the default trunk switch. For non-HA trunk links connected to Dual-NIC servers, use trunk server.

ServerIron(config)# vlan 2
ServerIron(config-vlan-2)# untag ethernet 13 to 14
ServerIron(config-vlan-2)# no spanning-tree
ServerIron(config-vlan-2)# exit
ServerIron(config)# server real web1 208.96.22.100
ServerIron(config-rs-web1)# port http
ServerIron(config-rs-web1)# port ssl
ServerIron(config-rs-web1)# port ftp
ServerIron(config-rs-web1)# port telnet
ServerIron(config-rs-web1)# server real web2 208.96.22.101
ServerIron(config-rs-web2)# port http
ServerIron(config-rs-web2)# port ssl
ServerIron(config-rs-web2)# port ftp
ServerIron(config-rs-web2)# port telnet
ServerIron(config-rs-web2)# server virtual www.alterego.com 208.96.6.254
ServerIron(config-vs-www.alterego.com)# port http
ServerIron(config-vs-www.alterego.com)# port ssl sticky
ServerIron(config-vs-www.alterego.com)# port ftp
ServerIron(config-vs-www.alterego.com)# port telnet
ServerIron(config-vs-www.alterego.com)# bind http web1 http web2 http
ServerIron(config-vs-www.alterego.com)# bind ssl web1 ssl web2 ssl
ServerIron(config-vs-www.alterego.com)# bind ftp web1 ftp web2 ftp
ServerIron(config-vs-www.alterego.com)# bind telnet web1 telnet web2 telnet
ServerIron(config-vs-www.alterego.com)# exit

To identify the router port, configure the trunk group, assign ports 13 and 14 as the backup ports, assign round robin as the predictor (load balancing metric), and disable Spanning Tree, enter the following commands:

ServerIron(config)# server router-ports 11
ServerIron(config)# server backup ethernet 13 00e0.5201.0c72
ServerIron(config)# server predictor round-robin
ServerIron(config)# no span
ServerIron(config)# exit
ServerIron# write memory
ServerIron# reload

The MAC address assigned is a MAC address that is resident on either SI1 or SI2. Notice that because port 13 is the lead port for the trunk group, you do not need to configure any other ports within that group.

Symmetric SLB

Overview

Symmetric SLB (SSLB) is active-standby VIP. Both SIs handle traffic, but the active VIP handles the L4-7 and the standby VIP serves only as a standby. Each SI is the active SI for a specific set of VIPs, while the other SI is the backup for the same set of VIPs.

In SSLB, you determine which SIs are active and backup for a VIP by associating a sym-priority with the VIP. You assign a different priority to the same VIP on each SI. The SI on which the VIP has the highest priority is the active SI for that VIP and the others are standbys. When all SIs and associated links are available, the SI with the highest priority for the VIP services the VIP.

SSLB does not require any changes to the Spanning Tree configuration in the network. Regardless of whether the network is using Spanning Tree, SSLB provides redundancy for the VIPs and allows all the SIs configured for SSLB to actively perform Server Load Balancing.

In addition, you do not need to dedicate SI links to SSLB. SSLB works within the network's topology.

Minimum Required Configuration

In the previous diagram, two upstream routers are connected to two different ISPs. This setup allows clients to access the SIs from different directions. Clients coming from ISP1 want an active VIP1 (on SI-A). The same VIP1 accessed by ISP2 is on standby (on SI-B). On a per SI basis, some VIPs are active while others are on standby. In contrast, all VIPs per SI are either active or standby in a Hot Standby scenario.

To configure symmetric SLB, configure the sym-priority <value> command on each active and standby VIP. The higher the <value>, the higher the preference (priority). The range is from 0 - 255. You also can configure the priority to dynamically adjust to changes in the health of applications on the VIP.

In the diagram, SI-A's VIP1 has a priority of 10. SI-B's VIP1 has a priority of 5. Therefore, SI-A is active. When traffic comes to VIP1, SI-A creates the session. When VIP1 on SI-A goes down, VIP1 on SI-B becomes active. Only the active VIP owner responds to ARP, traffic, session synching, and so on. The symmetric solution provides granular control of the VIPs.

Enabled by default, any L2 link can be used for automatic session synchronization between the SIs. Unlike Hot Standby, the SIs need not be directly connected. To specify a specific port (optional), use the session-sync server subcommand on both devices. See "Session-Sync".

Following is another common symmetric topology, where the real servers are directly connected to the SIs.

Failover Conditions

Both SIs are active with SSLB. Therefore, failover depends on which device has ownership of the VIP. If a link is broken, both SIs are still active. The only time a VIP can failover is during a reload or system crash. Use show log and show server virtual to gather failover information. The show server virtual command displays state information (5 = active, 3 = standby, 2 = inactive, 1 = inactive).

Session-Sync

For each port you use for load balancing, you must define the session-sync and port number to enable session synchronization.

ServerIron(config)# server port 80
ServerIron(config-port-80)# session-sync

This example enables session synchronization for port 80 (HTTP).

Syntax: server port <TCP/UDP-portnum>

Syntax: [no] session-sync

sym-pdu-rate

A SI in an SSLB configuration uses discovery packets to request SSLB information from the other SIs. SSLB discovery packets are proprietary Layer 2 broadcast packets and are sent on all ports in all port-based VLANs. Use server sym-pdu-rate to change the interval and wait time for SSLB discovery packets.

By default, a SI in an SSLB configuration sends discovery packets at 200-millisecond intervals. The SI will wait up to 20 equivalent intervals to receive a discovery packet from another SI. If the SI does not receive a discovery packet from the other SI within the 20 intervals, the SI concludes that its partner SI is unavailable and assumes control of the VIPs being managed by that SI. For example, if the interval for sending SSLB discovery packets is 200 milliseconds (the default), the SI will wait 20 x 200 milliseconds (four seconds) to receive a discovery packet from another SI.

You can change the discovery interval multiplier and the wait time multiplier.

The SSLB timer affects the rate at which the SI sends SSLB protocol packets to its SSLB partners. The timer does not affect client or server traffic to or from a VIP.

All the SIs in your configuration must use the same SSLB discovery interval and wait time. If you change the interval and wait time on one SI, make the same change on all the other SIs in the SSLB configuration.

[no] server sym-pdu-rate <disc-mult> <wait-time-mult>

<disc-mult>—specifies the multiplier for the SSLB protocol interval. You can specify a multiplier from 1 - 60. The default is 1.

<wait-time-mult>—specifies how many multiples of the discovery interval the SI will wait for an SSLB discovery packet. You can specify a multiplier from 1 - 60. The default is 20.

ServerIron(config)#server sym-pdu-rate 2 30

This command changes the interval at which the SI sends SSLB discovery packets to once every 400 milliseconds, and changes the maximum amount of time the SI will wait for an SSLB discovery packet from another SI to 12 seconds (30 x 400 milliseconds).

Dynamic Priority

Software release 07.1.08 enhances SSLB by automatically adjusting a VIP application's SSLB priority to a lower value if a given application fails a health check. With this enhancement, the SSLB priority provides failover for the individual application even if the SI and the application's VIP are both still active.

In previous releases, the priority determines which SI becomes the active one for the VIP and application by default. The priority is static and does not change if the status of the VIP's application changes. As a result, it is possible for SSLB to continue trying to use a real server farm that is no longer responding, instead of failing over to the other SI to load balance requests for the VIP and application.

In software release 07.1.08, you can configure a decrement value for the SSLB priority. If an application on a VIP that is enabled for SSLB fails a health check, the SI decrements the VIP's SSLB priority by the amount you specify for the decrement. If the priority value becomes lower than the VIP's priority on the other SI, the software fails the VIP over to the other SI.

Figure 8.5 shows an example of an SSLB configuration that uses the default priority handling (not the dynamic priority handling).

Using the default priority handling, the software fails over a VIP to the other SI only of the entire VIP or the SI itself becomes unavailable. If an application on the VIP becomes unavailable on all the real servers bound to the VIP, but the VIP itself is still available, the software continues using the same SI for the VIP. As a result, clients are unable to access the unavailable application even if the application is available through the other SI.

Figure 8.6 shows an example of a configuration that uses dynamic SSLB priority.

In this configuration, a SI fails over a VIP to the other SI if more than one application on the VIP becomes unavailable. If one application becomes unavailable, the software reduces the VIP's priority by 9 (the decrement value), in this case to 21. At this point, the SI that is active by default for the VIP still has the higher priority, so failover does not occur. However, if a second application becomes unavailable, then the priority becomes 12, which is less than the priority for the VIP on the other SI (20).

When an application becomes available again (and passes a health check), the SI increments the VIP's priority by the decrement amount, thus replacing the priority amount that the software removed when the application failed. If the increment makes the VIP's priority higher than the priority on the other SI, the software fails back over to the SI that originally had the higher priority for the VIP.

If more than one SI has the highest priority for a VIP, the SI that has the highest value for the lowest four bytes of its base MAC address becomes the active SI for the VIP.

Configuring Dynamic SSLB Priority

The following commands configure the SSLB priority parameters for the configuration in Figure 8.6.

Commands on SI A

SIA(config-vs-VIP1)# sym-priority 30
SIA(config-vs-VIP1)# dyn-sym-pri-factor 9

Commands on SI B

SIB(config-vs-VIP1)# sym-priority 20
SIB(config-vs-VIP1)# dyn-sym-pri-factor 9

The dyn-sym-pri-factor commands in this example configure the decrement value to 9. Each time an application on the VIP fails a health check (fails on all the real servers bound to the VIP), the SI decrements the VIP's SSLB priority by 9. If the priority reaches a value that is lower than the VIP's priority on the other SI, the software fails over active load balancing for the VIP to the other SI. In this example, failover of the VIP from SI A to SI B occurs if two applications are unavailable (have failed their health checks).

Syntax: [no] dyn-sym-pri-factor <num>

The <num> parameter can be a value from 0 - 255 and specifies the amount by which you want the SI to decrement a VIP's priority when an application on the VIP fails a health check. There is no default. If you specify a value, then the software performs dynamic SSLB priority for the VIP.

To remove a configured decrement, issue dyn-sym-pri-factor 0. The no form of the command does not disable the command.

Displaying Dynamic Priority Information

To display the dynamic SSLB configuration and current value for a VIP, enter a command such as the following at any level of the CLI:

Syntax: show server virtual [<name>]

This example shows the configuration and priority information for VIP1 in Figure 8.6. The priority information is shown by the fields in bold type. These fields show the following information.

delay-symmetric

Use server delay-symmetric to delay the reactivation of a failed SI in an SSLB configuration following the SI's recovery. By delaying reactivation of a recovered SI, you provide time for sessions created by the standby SI to terminate normally.

When you enable session synchronization in a SI SSLB configuration, the active SI for a VIP sends session synchronization information to the standby SI. If the VIP's active SI becomes unavailable, the open sessions for the VIP fail over to the other SI, which provides uninterrupted service for the sessions.

The active SI sends session synchronization information to a VIP's standby SI when the session is created. Following a failover, when the standby SI for a VIP has taken over, the standby SI can create new sessions for the VIP. However, because the SI with the higher priority for the VIP is unavailable, the standby SI cannot send synchronization information for the newly created sessions. As a result, when the other SI becomes available again, it resumes service for the VIP but cannot continue the sessions that were created by the standby SI.

You can minimize interruption to sessions created on the standby SI by configuring each SI to delay reactivation following its recovery after a failover. By delaying reactivation of a recovered SI, you provide time for sessions created by the standby SI to terminate normally.

[no] server delay-symmetric [<mins>]

The <mins> parameter specifies the number of minutes you want the recovered SI to wait before becoming active again. You can specify from 2 - 120 minutes. The default is 60 minutes. You must enter the same command using the same number of minutes on both SIs in the configuration.

ServerIron(config)#server delay-symmetric

show server symmetric

Use show server symmetric to display SSLB information:

ServerIron(config)# show server symmetric

Server Symmetric port = 0
Group_id = 1 Candidate cnt = 0
Port No-rx
1 100824

Table Of Contents | Previous Chapter | Next Chapter