December 24, 2013 12:17 a.m.
Ever run into a situation where a server has multiple nics that need failover if one fails or load balancing without switch or router level redundency? Or you have one nic with multiple ports distributed between two switches and the same needs? Linux network interface bonding can be your saviour!
Network bonding on Linux is like configuring multiple cards or ports as if they were the same physical device.
First and foremost you need to do away with NetworkManager if it is currently handling your network. You can do so with the following commands.
chkconfig NetworkManager off
service NetworkManager stop
systemctl disable NetworkManager.service
Then we need to tell the system about bonding and load the proper kernel modules. Edit or create the /etc/modprobe.d/bonding.conf file if it does not currently exist and add these lines. The second line is optional here and you can also add it on the /etc/sysconfig/network-scripts/ifcfg-bond0 interface file as I will show later.
alias netdev-bond0 bonding
options bond0 mode=0 miimon=1000
Now we need to configure the actual network interfaces and the bonding interface. I am using eth0, eth1 and bond0 as examples. Please substitue your actual interface names as needed. All three interfaces are configured by editing /etc/sysconfig/network-scripts/ifcfg-ethX or /etc/sysconfig/network-scripts/bondX.
- Mode 0 (balance-rr)
This mode transmits packets in a sequential order from the first available slave through the last. If two real interfaces are slaves in the bond and two packets arrive destined out of the bonded interface the first will be transmitted on the first slave and the second frame will be transmitted on the second slave. The third packet will be sent on the first and so on. This provides load balancing and fault tolerance.
- Mode 1 (active-backup)
This mode places one of the interfaces into a backup state and will only make it active if the link is lost by the active interface. Only one slave in the bond is active at an instance of time. A different slave becomes active only when the active slave fails. This mode provides fault tolerance.
- Mode 2 (balance-xor)
Transmits based on XOR formula. (Source MAC address is XOR’d with destination MAC address) modula slave count. This selects the same slave for each destination MAC address and provides load balancing and fault tolerance.
- Mode 3 (broadcast)
This mode transmits everything on all slave interfaces. This mode is least used (only for specific purpose) and provides only fault tolerance.
- Mode 4 (802.3ad)
This mode is known as Dynamic Link Aggregation mode. It creates aggregation groups that share the same speed and duplex settings. This mode requires a switch that supports IEEE 802.3ad Dynamic link.
- Mode 5 (balance-tlb)
This is called as Adaptive transmit load balancing. The outgoing traffic is distributed according to the current load and queue on each slave interface. Incoming traffic is received by the current slave.
- Mode 6 (balance-alb)
This is Adaptive load balancing mode. This includes balance-tlb + receive load balancing (rlb) for IPV4 traffic. The receive load balancing is achieved by ARP negotiation. The bonding driver intercepts the ARP Replies sent by the server on their way out and overwrites the src hw address with the unique hw address of one of the slaves in the bond such that different clients use different hw addresses for the server
I personally have run into issues with vlan tagging and bonding in mode 0 (round robin). I have worked around the issue with using mode 1 (active backup) instead. Another post will tackle those configurations and points that should be noted.