Static routes disappear after a reboot: make them persistent

You’ll add one practice route, 10.70.0.0/16 via 10.10.10.1, on both VMs: first the quick runtime way, so you learn to spot it, then the persistent way, proved with a reboot. In the lab the next hop is the same NAT gateway as the default route, so traffic doesn’t change path; at work it would be a separate router to a partner or a backup network. The 10.50.0.0/16 routes from Lessons 31 and 32 stay as they are.

Take snapshots first (VBoxManage snapshot rocky01 take before-l69 and the same for ubu01).

1Read where each route on rocky01 came from

Run this on rocky01:

ip route
ip route show proto boot | wc -l

You should see something like this:

default via 10.10.10.1 dev enp0s3 proto static metric 100
10.10.10.0/24 dev enp0s3 proto kernel scope link src 10.10.10.11 metric 100
10.10.20.0/24 dev enp0s8 proto kernel scope link src 10.10.20.11 metric 101
10.10.30.0/24 dev bond0 proto kernel scope link src 10.10.30.11 metric 300
10.50.0.0/16 via 10.10.10.1 dev enp0s3 proto static metric 100
0

Every line has a proto word. The two kernel routes on enp0s8 and bond0 come from the storage network of Lesson 48 and the bond of Lesson 31, and the default and 10.50.0.0/16 routes come from the enp0s3 profile, tagged static by NetworkManager. Zero lines for proto boot means nobody has added anything by hand since the last boot.

✅ Checkpoint: Every route shows a proto word and the proto boot count is 0.

2Add a route the incident way and catch it

Run this:

sudo ip route add 10.70.0.0/16 via 10.10.10.1
ip route show 10.70.0.0/16
ip route show proto boot
ip -d route show 10.70.0.0/16

You should see this:

10.70.0.0/16 via 10.10.10.1 dev enp0s3
10.70.0.0/16 via 10.10.10.1 dev enp0s3
unicast 10.70.0.0/16 via 10.10.10.1 dev enp0s3 proto boot scope global

The route works right now, and it’s the only line with no proto word, which is how you spot it in a long table. ip -d (details) prints the hidden tag: proto boot. It also has no metric, which means metric 0, the strongest possible, so it will keep winning over any configured copy you add next.

✅ Checkpoint: ip route show proto boot lists one route, 10.70.0.0/16.

3Make it persistent with nmcli

Add it to the enp0s3 profile with metric 50, then remove the runtime copy and apply:

sudo nmcli connection modify enp0s3 +ipv4.routes "10.70.0.0/16 10.10.10.1 50"
sudo ip route del 10.70.0.0/16
sudo nmcli device reapply enp0s3
ip route show 10.70.0.0/16
nmcli -g ipv4.routes connection show enp0s3
sudo grep '^route' /etc/NetworkManager/system-connections/enp0s3.nmconnection

You should see something like this:

Connection successfully reapplied to device 'enp0s3'.
10.70.0.0/16 via 10.10.10.1 dev enp0s3 proto static metric 50
10.50.0.0/16 10.10.10.1, 10.70.0.0/16 10.10.10.1 50
route1=10.50.0.0/16,10.10.10.1
route2=10.70.0.0/16,10.10.10.1,50

The third number in the nmcli route is the metric, stored as the third field of route2=. Deleting the hand-made copy matters: had you left it, ip route show 10.70.0.0/16 would list two routes and the kernel would keep using the metric 0 one, so your check after the reboot would be testing the wrong thing.

Try it yourself: build the persistent nmcli route command, with a metric, in the Command Builder.

✅ Checkpoint: One 10.70.0.0/16 route, tagged proto static metric 50, and a route2= line in the keyfile.

4Write a route check you can run after every reboot

Paste this whole block:

mkdir -p ~/lab69
cat > ~/lab69/route-check.sh <<'EOF'
#!/bin/bash
# route-check.sh: check that each route this server needs is in the kernel and will survive a reboot.
# SaatVedha Linux Administration, Lesson 69. Each line of the list: destination next-hop
LIST=${1:-$HOME/lab69/routes.txt}
fail=0
while read -r dest want; do
    [[ -z $dest || $dest == \#* ]] && continue
    line=$(ip -4 route show exact "$dest" | head -n 1)
    got=$(awk '{for (i = 1; i < NF; i++) if ($i == "via") { print $(i + 1); exit }}' <<< "$line")
    if [[ -z $line ]]; then
        echo "FAIL  $dest is missing, traffic falls back to the default route"; fail=1
    elif [[ $got != "$want" ]]; then
        echo "FAIL  $dest goes via ${got:-no gateway}, expected $want"; fail=1
    elif [[ $line != *proto* ]]; then
        echo "WARN  $dest via $got is runtime only (proto boot), a reboot will remove it"; fail=1
    else
        echo "OK    $dest via $got"
    fi
done < "$LIST"
exit "$fail"
EOF
chmod +x ~/lab69/route-check.sh
printf '# destination   next-hop\n10.50.0.0/16 10.10.10.1\n10.70.0.0/16 10.10.10.1\n' > ~/lab69/routes.txt
~/lab69/route-check.sh; echo "exit=$?"

You should see this:

OK    10.50.0.0/16 via 10.10.10.1
OK    10.70.0.0/16 via 10.10.10.1
exit=0

For each destination in routes.txt, the script asks for that exact route with ip -4 route show exact, checks the next hop, and flags a route with no proto word as runtime only. A plain ip route get wouldn’t do here, because in the lab the default route uses the same gateway and would hide a missing route. At work you’d keep routes.txt in your configuration repository and run the script from the post-patching checklist or monitoring.

✅ Checkpoint: Both routes print OK and the exit code is 0.

5Reboot rocky01 and prove it

Run this, wait a minute, then log in again:

sudo systemctl reboot

After the reboot, run this:

ip route show 10.70.0.0/16
ip route get 10.70.4.20
~/lab69/route-check.sh; echo "exit=$?"

You should see something like this:

10.70.0.0/16 via 10.10.10.1 dev enp0s3 proto static metric 50
10.70.4.20 via 10.10.10.1 dev enp0s3 src 10.10.10.11 uid 1000
    cache
OK    10.50.0.0/16 via 10.10.10.1
OK    10.70.0.0/16 via 10.10.10.1
exit=0

The route came back from the keyfile at boot. ip route get shows the full decision: the gateway, the interface, the source address the server will use and your UID, because you asked as student. The “cache” line only means the answer came from the kernel’s route lookup; there is no separate cache to clear on modern kernels.

✅ Checkpoint: After the reboot, route-check.sh prints two OK lines on rocky01.

6Add the same route on ubu01 with netplan

On ubu01, back up the file, then write it with the new route (the rest is the Lesson 32 file):

sudo cp /etc/netplan/50-cloud-init.yaml /root/50-cloud-init.yaml.l69
sudo tee /etc/netplan/50-cloud-init.yaml > /dev/null <<'EOF'
network:
  version: 2
  ethernets:
    enp0s3:
      dhcp4: false
      addresses:
        - 10.10.10.21/24
      routes:
        - to: default
          via: 10.10.10.1
        - to: 10.50.0.0/16
          via: 10.10.10.1
        - to: 10.70.0.0/16
          via: 10.10.10.1
          metric: 50
      nameservers:
        search: [saatlab.test]
        addresses: [10.10.10.1]
EOF
sudo netplan generate && sudo grep -A3 '10.70' /run/systemd/network/10-netplan-enp0s3.network

You should see this:

Destination=10.70.0.0/16
Gateway=10.10.10.1
Metric=50

netplan generate checks the YAML and writes the networkd file without touching the network yet. Each item under routes: becomes a [Route] section; metric: becomes Metric=. tee kept the file’s mode 600, so netplan doesn’t warn about permissions. Your 60-storage.yaml from Lesson 48 is a separate file and isn’t affected.

Try it yourself: build a netplan file with extra routes in the Command Builder.

✅ Checkpoint: The generated .network file has a [Route] for 10.70.0.0/16 with Metric=50.

7Apply with netplan try and prove it after a reboot

Run this, then press Enter when the countdown starts:

sudo netplan try

You should see something like this:

Do you want to keep these settings?


Press ENTER before the timeout to accept the new configuration


Changes will revert in 112 seconds
Configuration accepted.

Copy the check script from rocky01, reboot ubu01 and check:

mkdir -p ~/lab69
scp rocky01:lab69/route-check.sh rocky01:lab69/routes.txt ~/lab69/
sudo systemctl reboot

After the reboot:

ip route show proto static
~/lab69/route-check.sh; echo "exit=$?"

You should see something like this:

default via 10.10.10.1 dev enp0s3 proto static
10.50.0.0/16 via 10.10.10.1 dev enp0s3 proto static
10.70.0.0/16 via 10.10.10.1 dev enp0s3 proto static metric 50
OK    10.50.0.0/16 via 10.10.10.1
OK    10.70.0.0/16 via 10.10.10.1
exit=0

scp uses the student key from Lesson 33, so no password is asked. networkd tags its routes proto static too, so the same script works on both distributions without a change. That’s the point of checking the kernel instead of the config files: the files differ, the table doesn’t.

✅ Checkpoint: ubu01 shows 10.70.0.0/16 proto static metric 50 after the reboot and the script exits with 0.