Network dynamic failover with bonding - how to have ethernet interface detect that internet is down (lan is still up)?

Question

after reading this answer from @Ingo for how to setup dynamic network failover I am having a question. First of all thanks a lot to @Ingo posting this manual how to setup bonding/dynamic network failover. It was very easy to setup and works fine for me if my ethernet and wlan network IP address are the same!

My goal is to have the interface switch over via bonding to the wlan interface (that is a LTE/cellular router) if the internet connection on the ethernet interface is down.

Currently the bonding failover only works if I fully disconnect the ethernet (by unplugging the cable). However, I would like the ethernet interface to set the MII status to down if the internet is not working anymore.

Does somebody have a tip here for me how to approach this? Thanks a lot in advance!

Answer 1

@goldilocks put it in a nutshell in his comment so I will quote it here:

There's no way to do that passively (ie., without sending packets out), and no definitive universal way to do it actively by noting a failure to send, since that is very context dependent (send what where? Eg., Does a failure to connect to google.com:80 indicate the internet is down? And detecting a failure to connect to anything would take a very, very long time). The router can tell when the uplink is gone, but it will not turn off the LAN because of that even if there's only one thing connected.

But you asked (in a reply to this) if there is a way to have a local logic implemented that for example pings google every 2mins and if not reachable, failover?

This is not an easy task and we have to grab deep into the bag of tricks about networking. There are mainly two issues:
1. The eth0 interface must always be up. Otherwise we can't ping if the connection is back again. We can only manage that eth0 is a slave member of bonding or not.
2. It is the nature of bonding that you always have a connection, no matter if eth0 is connected or not. So you can't simply ping from eth0. You will always get a response because the default route is going through interface bond0. So we have to use source routing to see if a connection from interface eth0 (not a slave of bonding at this time) succeeds. This is made with policy routing but details about it are out of scope here. Just take it.

Here is a tested example how I would do it. To simplify things I have eth0 given a static ip address. You have already made a setup as shown in Howto migrate from networking to systemd-networkd with dynamic failover.

First create a new routing table pingtest with ID 200:

rpi ~$ sudo bash -c 'echo 200 pingtest >> /etc/iproute2/rt_tables'

Then ensure that /etc/systemd/network/12-bond0-add-eth.network looks like this:

[Match]
Name=e*

[Network]
Bond=bond0
PrimarySlave=yes
Address=192.168.50.60/24

[RoutingPolicyRule]
Table=200
Priority=16384
From=192.168.50.60

[Route]
# This makes a default route
Table=200
Destination=0.0.0.0/0
Protocol=static
Gateway=192.168.50.1
PreferredSource=192.168.50.60

Then create a bash script pingtest.sh:

#!/bin/bash
IF="eth0"
BOND="bond0"
PING_SOURCE="192.168.50.60"
PING_DEST="google.com"
POLL="120"   # polling time in seconds

while true; do
    if [[ $(/bin/ip -br link show dev "$IF") == *",SLAVE,UP,"* ]]; then
        #echo DEBUG: "$IF" is slave
        /bin/ping -Bnq -c3 -w3 "$PING_DEST" &>/dev/null
        if [ $? -ne 0 ]; then
            /bin/ip link set "$IF" nomaster
            /bin/ip link set "$IF" up
            echo "$IF" removed from "$BOND", ping to "$PING_DEST" failed
        fi
    else
        #echo DEBUG: "$IF" is no slave
        /bin/ping -Bnq -c3 -w3 -I "$PING_SOURCE" "$PING_DEST" &>/dev/null
        if [ $? -eq 0 ]; then
            /bin/ip link set "$IF" down
            /bin/ip link set "$IF" master "$BOND"
            /bin/ip link set "$IF" up
            echo "$IF" added to "$BOND", ping to "$PING_DEST" succeeded
        fi
    fi
    sleep "$POLL"
done

For testing you can uncomment the two echo DEBUG: statements and execute the script from the command line with sudo. Don't forget to make it executable with chmod +x pingtest.sh. Messages from the script about bond changes you will find in the journal or with systemctl status pingtest.service (service see below).

To run the script create a service with:

rpi ~$ sudo systemctl --force --full edit pingtest.service

In the empty editor insert these statements, save them and quit the editor:

[Unit]
Description=Pingtest if destination is up
Wants=network.target
After=network.target

[Service]
ExecStart=/home/pi/pingtest.sh

[Install]
WantedBy=network.target

Enable the new service with

rpi ~$ sudo systemctl enable pingtest.service

Reboot.

The routing of this setup looks like this:

rpi ~$ ip route show table main
default via 192.168.50.1 dev bond0 proto dhcp src 192.168.50.205 metric 1024 
192.168.50.0/24 dev bond0 proto kernel scope link src 192.168.50.205 
192.168.50.0/24 dev eth0 proto kernel scope link src 192.168.50.60 
192.168.50.1 dev bond0 proto dhcp scope link src 192.168.50.205 metric 1024

rpi ~$ ip route show table pingtest
default via 192.168.50.1 dev eth0 proto static src 192.168.50.60

rpi ~$ ip rule ls
0:      from all lookup local 
16384:  from 192.168.50.60 lookup pingtest 
32766:  from all lookup main 
32767:  from all lookup default