Static IP, routes and bonds with nmcli on RHEL and Rocky 10
Setting an nmcli static IP is how you give a Red Hat family server (RHEL, Rocky, AlmaLinux, Oracle Linux) a fixed address, gateway, DNS servers and routes that come back the same after every reboot. NetworkManager owns the network on these systems, nmcli is its command line, and since RHEL 10 its keyfiles are the only place that configuration lives: the old ifcfg files and the teamd service are gone.
In Lesson 30 you read the network with ip and saw that ip changes vanish at reboot. Here you make permanent changes on rocky01: you read the keyfile behind enp0s3, set its address and DNS, add a second address and a static route to 10.50.0.0/16, and then add two network cards on the VirtualBox internal network “bondnet” and join them into bond0, an active-backup bond with 10.10.30.11/24 that keeps working when one card loses its link.
Where this lesson fits
Lesson 31 of 82 in Linux Administration · Phase 2: Day-to-day administration and the command builder · Before this: Lesson 30, Linux networking basics · Next: Lesson 32, static IP and routes with netplan on Ubuntu
| Time | You need | By the end you can |
|---|---|---|
| About 75 minutes | rocky01 from Lesson 2, the VirtualBox console window for rocky01, and VBoxManage on the laptop (rocky01 is shut down once, in step 7) | Read and change NetworkManager connections with nmcli, explain the keyfile, add persistent addresses and routes, and build and test an active-backup bond |
What it is, in one line
nmcli static IP configuration means writing the address, gateway, DNS and routes into a NetworkManager connection, which is saved as a keyfile and applied to the device every time it comes up.
In Simple Words
The tailor’s measurement book
A good tailor keeps a measurement book with a page for each customer: chest, sleeve, length, the colour of thread they like. When you come in, he doesn’t measure you from scratch; he opens your page and cuts the cloth to it. If he pins the sleeve shorter during a fitting but never writes it on the page, the next shirt comes out with the old sleeve.
NetworkManager works the same way. The connection is the page in the book (saved as a keyfile), the device is the customer standing in the shop, and changes made with the ip command are pins that come out at the next fitting.
Engineer's View
Devices, connections, keyfiles, and what RHEL 10 removed
NetworkManager separates two things that older tools mixed up. A device is a real or virtual network interface the kernel has, such as enp0s3 or bond0. A connection (also called a profile) is a saved set of settings: addresses, gateway, DNS, routes, bond options.
A device has at most one active connection at a time, and one device can have several connections to choose from (office and home Wi-Fi on a laptop).
| Thing | Where it lives | Command |
|---|---|---|
| Connection profile | A keyfile: /etc/NetworkManager/system-connections/NAME.nmconnection, root only, mode 600 | nmcli connection show, add, modify, delete |
| Applying a profile | NetworkManager pushes it to the kernel | nmcli connection up NAME (re-activates), nmcli device reapply DEV (applies changes in place) |
| Device state now | The kernel, as NetworkManager sees it | nmcli device status, nmcli device show DEV |
| One setting, for scripts | The profile | nmcli -g ipv4.addresses connection show NAME |
| Menu-driven editor | Same profiles | nmtui (text menus, handy on a console) |
Properties have the form setting.property, such as ipv4.addresses or bond.options. For list properties, a plus sign adds an item (+ipv4.routes), a minus sign removes one, and no sign replaces the whole list. nmcli connection modify only changes the saved profile; nothing happens on the wire until you run connection up or device reapply.
| Older RHEL habit | On RHEL 10, Rocky 10, Oracle Linux 10 |
|---|---|
/etc/sysconfig/network-scripts/ifcfg-eth0 | Support for ifcfg files is removed. Keyfiles only (new profiles have been keyfiles since RHEL 9). On a RHEL 9 server, nmcli connection migrate converts ifcfg profiles before an upgrade |
route-eth0 files for static routes | +ipv4.routes on the connection (route-<if> files were for RHEL 7 and 8) |
ifup / ifdown from network-scripts | nmcli connection up / down |
| Network teaming (teamd, libteam) | Removed. Use kernel bonding; migrate any team to a bond before upgrading |
dhclient | Removed. NetworkManager’s internal DHCP client is the default |
How it works, step by step
Follow what happens when you change rocky01’s settings with nmcli. The diagram below shows the three places a setting can be, and which command moves it where.
1nmcli modify writes the profile
nmcli connection modify enp0s3 ... asks NetworkManager to change the connection named enp0s3. NetworkManager checks the values (a gateway with no address is refused, a bad route is refused) and writes the keyfile in /etc/NetworkManager/system-connections. The running interface is not touched yet.
2up or reapply pushes it to the device
nmcli connection up enp0s3 re-activates the profile on its device: the link is reconfigured from scratch, which drops existing connections for a moment. nmcli device reapply enp0s3 applies the changed settings to the running device without taking it down, and works for most IP changes (addresses, routes, DNS).
3At boot, the keyfile is the truth
When NetworkManager starts, it loads every keyfile and activates the ones with autoconnect on. Anything you did with ip addr add or ip route add was never in a keyfile, so it doesn’t come back. That’s why a route that “worked yesterday” can disappear after patching night, which is Break-It 1 below and Lesson 69 in full.
4Static routes belong to the connection
A route like 10.50.0.0/16 10.10.10.1 (destination, then next hop, no “via”) is stored as a route1= line in the keyfile and added each time the connection comes up. NetworkManager also removes it when the connection goes down, so the route always follows the interface it belongs to.
5A bond is one connection with ports
For a bond you create three connections: bond0 itself (type bond, with the IP address and bond.options), and one ethernet connection per card with port-type bond and controller bond0. The ports carry no IP settings; the address lives only on bond0. Older guides write the same thing as slave-type bond master bond0, which still works.
In active-backup mode (mode 1) one port carries all traffic and the other waits. The kernel checks each port’s link every miimon=100 milliseconds, and when the active port loses its link it moves the bond’s MAC address to the backup port and sends gratuitous ARPs so switches learn the new path. No switch configuration is needed, which is why it is the safe default; 802.3ad (mode 4, LACP) uses both links at once but needs the switch ports set up as a LACP group.
The picture below shows the bond you’ll build.
6Check it the kernel’s way
nmcli tells you what NetworkManager wanted. /proc/net/bonding/bond0 tells you what the bonding driver is doing right now: the mode, which port is active, each port’s link status and how many times each one has failed. During an incident, read that file before anything else.
Make one change at a time, and for anything that touches the interface you are connected through, work from the console. On a real server that means the out-of-band console (iLO, iDRAC, a hypervisor console), opened and logged in before you press Enter.
Hands-on lab: static IP, a persistent route and an active-backup bond on rocky01
Before you run any command, double-check it
Read every command twice before you press Enter. Check the target: the server name, the user, the disk or device name, the path and the sizes. These commands are for learning and were tested in our lab, not on your system. At work, run the read-only commands first, follow your change process, make sure a backup exists, and ask a second person to check anything that changes or deletes data. You run them at your own risk.
Everything runs on rocky01, with two VBoxManage commands on the laptop in steps 7 and 9. Open rocky01’s VirtualBox console window now and log in there as student: steps 4 and 6 bring enp0s3 down and up, and you want to be on the console, not on SSH through that same card, when they do.
1Tell devices and connections apart
Run this:
nmcli device status
nmcli connection showYou should see something like this:
DEVICE TYPE STATE CONNECTION
enp0s3 ethernet connected enp0s3
lo loopback connected (externally) lo
NAME UUID TYPE DEVICE
enp0s3 5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03 ethernet enp0s3
lo 2d5bd0a4-1e26-4b0e-9c3e-6a1f7d2c8b44 loopback loThe first table is devices (interfaces the kernel has) and which connection each is using. The second is connections (saved profiles). The Rocky installer named the profile enp0s3, the same as the device, which is convenient but can hide the difference: commands that take a NAME after connection want the profile name, commands after device want the interface.
Recent NetworkManager also shows the loopback lo, which you leave alone.
✅ Checkpoint: You can point at the device column and the connection column and say which is which.
2Read the keyfile behind the connection
Run this:
sudo ls -l /etc/NetworkManager/system-connections/
sudo cat /etc/NetworkManager/system-connections/enp0s3.nmconnection
ls /etc/sysconfig/network-scriptsYou should see something like this:
total 4
-rw-------. 1 root root 266 Oct 20 10:31 enp0s3.nmconnection
ls: cannot access '/etc/sysconfig/network-scripts': No such file or directory
[connection]
id=enp0s3
uuid=5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03
type=ethernet
interface-name=enp0s3
[ethernet]
[ipv4]
address1=10.10.10.11/24
dns=10.10.10.1;
dns-search=saatlab.test;
gateway=10.10.10.1
method=manual
[ipv6]
addr-gen-mode=eui64
method=auto
[proxy]A keyfile is a plain INI file: sections in brackets, one setting per line. address1 is the first address, gateway and DNS sit in the ipv4 section, and the uuid is what NetworkManager uses internally. The file must be owned by root with mode 600; with looser permissions NetworkManager refuses to load it. There is no network-scripts folder on Rocky 10, so any guide that tells you to edit ifcfg-enp0s3 is written for an older release.
You may edit a keyfile with vim, but then run sudo nmcli connection reload so NetworkManager reads it again, and nmcli connection up to apply it. nmcli is safer, because it checks every value before writing.
✅ Checkpoint: You found address1, gateway and dns in the ipv4 section of enp0s3.nmconnection.
3Read single settings with nmcli -g
Run this:
nmcli -g ipv4.method,ipv4.addresses,ipv4.gateway,ipv4.dns,ipv4.dns-search connection show enp0s3
nmcli -f GENERAL.STATE,IP4.ADDRESS,IP4.GATEWAY,IP4.DNS device show enp0s3You should see something like this:
manual
10.10.10.11/24
10.10.10.1
10.10.10.1
saatlab.test
GENERAL.STATE: 100 (connected)
IP4.ADDRESS[1]: 10.10.10.11/24
IP4.GATEWAY: 10.10.10.1
IP4.DNS[1]: 10.10.10.1-g (get values) prints only the values, one per line in the order you asked, which is what you want in a script or a health check. The second command shows the device side: what is applied right now, in capital letters. When the two disagree, someone changed the profile and didn’t apply it, or changed the device with ip and didn’t save it.
✅ Checkpoint: The connection and the device both report 10.10.10.11/24, gateway 10.10.10.1 and DNS 10.10.10.1.
4Set the static IP with nmcli, from the console
In the VirtualBox console window, run:
sudo nmcli connection modify enp0s3 ipv4.method manual ipv4.addresses 10.10.10.11/24 ipv4.gateway 10.10.10.1 ipv4.dns "10.10.10.1" ipv4.dns-search saatlab.test
sudo nmcli connection up enp0s3
ip -br address
ip routeYou should see something like this:
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/4)
lo UNKNOWN 127.0.0.1/8 ::1/128
enp0s3 UP 10.10.10.11/24 fe80::a00:27ff:fe3b:9c11/64
default via 10.10.10.1 dev enp0s3 proto static metric 100
10.10.10.0/24 dev enp0s3 proto kernel scope link src 10.10.10.11 metric 100This is the complete static IP command you’ll type on any RHEL family server: method manual, address with its prefix, gateway, DNS servers (space separated inside the quotes for more than one) and the search domain. On rocky01 the values match what the installer set in Lesson 2, so nothing moves, which is a good way to practise the command safely. connection up re-activated the profile; any SSH session through enp0s3 would have paused for a second.
Try it yourself: build the nmcli static IP commands for your own connection name and addresses in the Command Builder.
✅ Checkpoint: ip shows 10.10.10.11/24 on enp0s3 and default via 10.10.10.1.
5Add and remove a second address with + and –
Run this:
sudo nmcli connection modify enp0s3 +ipv4.addresses 10.10.10.111/24
sudo nmcli device reapply enp0s3
ip -br address show enp0s3
nmcli -g ipv4.addresses connection show enp0s3You should see something like this:
Connection successfully reapplied to device 'enp0s3'.
enp0s3 UP 10.10.10.11/24 10.10.10.111/24 fe80::a00:27ff:fe3b:9c11/64
10.10.10.11/24, 10.10.10.111/24The plus sign added 10.10.10.111 to the list instead of replacing it, and device reapply put it on the card without bouncing the link. A second address like this is how a service IP moves between servers, or how you prepare a new address before switching DNS to it. Now take it away again:
sudo nmcli connection modify enp0s3 -ipv4.addresses 10.10.10.111/24
sudo nmcli device reapply enp0s3
ip -br address show enp0s3You should see something like this:
Connection successfully reapplied to device 'enp0s3'.
enp0s3 UP 10.10.10.11/24 fe80::a00:27ff:fe3b:9c11/64✅ Checkpoint: enp0s3 is back to the single address 10.10.10.11/24.
6Add a static route that survives a reboot
Still on the console, run:
sudo nmcli connection modify enp0s3 +ipv4.routes "10.50.0.0/16 10.10.10.1"
sudo nmcli connection up enp0s3
ip route show 10.50.0.0/16
nmcli -g ipv4.routes connection show enp0s3
sudo grep '^route' /etc/NetworkManager/system-connections/enp0s3.nmconnectionYou should see something like this:
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/5)
10.50.0.0/16 via 10.10.10.1 dev enp0s3 proto static metric 100
10.50.0.0/16 10.10.10.1
route1=10.50.0.0/16,10.10.10.1The route is in the kernel, in the profile and in the keyfile as route1=. In the lab the next hop is the same 10.10.10.1 as the default gateway, so traffic doesn’t change path; at work it would be a different router, such as the one in front of a partner’s network or a backup VLAN. Now reboot and check it came back:
sudo reboot
# after logging in again:
ip route show 10.50.0.0/1610.50.0.0/16 via 10.10.10.1 dev enp0s3 proto static metric 100Try it yourself: build the persistent route command in the Command Builder.
✅ Checkpoint: After the reboot, ip route still shows 10.50.0.0/16 via 10.10.10.1.
7Add two network cards on the bondnet network
Shut rocky01 down, then run this on the laptop (one line, PowerShell or a terminal):
VBoxManage modifyvm rocky01 --nic3 intnet --intnet3 bondnet --nictype3 82540EM --nicpromisc3 allow-all --nic4 intnet --intnet4 bondnet --nictype4 82540EM --nicpromisc4 allow-allThat gives rocky01 NIC3 and NIC4 on a VirtualBox internal network called bondnet, which only VMs can see, with the same Intel card type as NIC1. Promiscuous mode “allow-all” lets a card accept frames for the bond’s MAC address after a failover, which a real switch port does anyway. Start rocky01 and log in on the console:
nmcli device statusYou should see something like this:
DEVICE TYPE STATE CONNECTION
enp0s3 ethernet connected enp0s3
lo loopback connected (externally) lo
enp0s9 ethernet disconnected --
enp0s10 ethernet disconnected --If you see profiles named “Wired connection 1” and “Wired connection 2” on enp0s9 and enp0s10 (NetworkManager makes them for new cards when no-auto-default isn’t set), delete them first with sudo nmcli connection delete "Wired connection 1" and the same for 2, so they don’t fight the bond ports.
[Add your screenshot here]: rocky01’s Settings, Network, Adapter 3 in VirtualBox Manager showing “Attached to: Internal Network”, Name “bondnet”, Promiscuous Mode “Allow All”.
✅ Checkpoint: enp0s9 and enp0s10 appear in nmcli device status.
8Build bond0 in active-backup mode
Paste this whole block:
sudo nmcli connection add type bond con-name bond0 ifname bond0 bond.options "mode=active-backup,miimon=100" connection.autoconnect-ports 1 ipv4.method manual ipv4.addresses 10.10.30.11/24 ipv6.method disabled
sudo nmcli connection add type ethernet con-name bond0-port1 ifname enp0s9 port-type bond controller bond0
sudo nmcli connection add type ethernet con-name bond0-port2 ifname enp0s10 port-type bond controller bond0
sudo nmcli connection up bond0
nmcli connection show
ip -br address show bond0You should see something like this:
Connection 'bond0' (0d33caba-3ef7-4d6b-861a-725a8f2f39e8) successfully added.
Connection 'bond0-port1' (32ce87e2-4146-4233-9f1e-828a5c51e5ad) successfully added.
Connection 'bond0-port2' (a07e0c45-93aa-444e-a370-57a72f22c4c4) successfully added.
Connection successfully activated (master waiting for slaves) (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/8)
NAME UUID TYPE DEVICE
enp0s3 5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03 ethernet enp0s3
bond0 0d33caba-3ef7-4d6b-861a-725a8f2f39e8 bond bond0
bond0-port1 32ce87e2-4146-4233-9f1e-828a5c51e5ad ethernet enp0s9
bond0-port2 a07e0c45-93aa-444e-a370-57a72f22c4c4 ethernet enp0s10
lo 2d5bd0a4-1e26-4b0e-9c3e-6a1f7d2c8b44 loopback lo
bond0 UP 10.10.30.11/24bond0 got the IP address and the bond options; the two port profiles only say which card joins which bond. connection.autoconnect-ports 1 makes bringing up bond0 also bring up its ports, and it’s the reason the activation line says the bond was waiting for its ports. Now ask the kernel what it built:
cat /proc/net/bonding/bond0
sudo cat /etc/NetworkManager/system-connections/bond0.nmconnectionYou should see something like this:
Ethernet Channel Bonding Driver: v6.12.0-211.16.1.el10_2.0.1.x86_64
Bonding Mode: fault-tolerance (active-backup)
Primary Slave: None
Currently Active Slave: enp0s9
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Peer Notification Delay (ms): 0
Slave Interface: enp0s9
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 08:00:27:c4:1e:09
Slave queue ID: 0
Slave Interface: enp0s10
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 08:00:27:7b:52:aa
Slave queue ID: 0
[connection]
id=bond0
uuid=0d33caba-3ef7-4d6b-861a-725a8f2f39e8
type=bond
autoconnect-ports=1
interface-name=bond0
[bond]
miimon=100
mode=active-backup
[ipv4]
address1=10.10.30.11/24
method=manual
[ipv6]
addr-gen-mode=default
method=disabled
[proxy]The mode is active-backup, enp0s9 is the active port, both ports have their link (MII Status: up) and neither has failed yet. “Slave” is the bonding driver’s old word for a port, and it still appears in this file. Write down the two failure counts; the next step changes one of them.
✅ Checkpoint: /proc/net/bonding/bond0 shows active-backup, Currently Active Slave enp0s9, and both ports with MII Status up.
9Fail a link and watch the backup take over
On the laptop, unplug NIC3’s virtual cable:
VBoxManage controlvm rocky01 setlinkstate3 offThen on rocky01:
grep -E 'Currently Active|Slave Interface|MII Status|Failure Count' /proc/net/bonding/bond0
sudo journalctl -k --since "2 min ago" --no-pager | grep -E 'enp0s9|bond0'
ip -br address show bond0You should see something like this:
Currently Active Slave: enp0s10
MII Status: up
Slave Interface: enp0s9
MII Status: down
Link Failure Count: 1
Slave Interface: enp0s10
MII Status: up
Link Failure Count: 0
Oct 27 14:21:07 rocky01 kernel: e1000: enp0s9 NIC Link is Down
Oct 27 14:21:07 rocky01 kernel: bond0: (slave enp0s9): link status definitely down, disabling slave
Oct 27 14:21:07 rocky01 kernel: bond0: (slave enp0s10): making interface the new active one
bond0 UP 10.10.30.11/24Within about 100 ms the kernel saw the link go down, disabled enp0s9 and made enp0s10 active. bond0 kept its address the whole time. There is no other machine on bondnet in this lab, so the proof is the driver’s own view; with a second VM on bondnet, a ping running during the test would lose at most one or two replies.
Plug the cable back in on the laptop, then check rocky01 again:
VBoxManage controlvm rocky01 setlinkstate3 ongrep -E 'Currently Active|Slave Interface|MII Status|Failure Count' /proc/net/bonding/bond0You should see something like this:
Currently Active Slave: enp0s10
MII Status: up
Slave Interface: enp0s9
MII Status: up
Link Failure Count: 1
Slave Interface: enp0s10
MII Status: up
Link Failure Count: 0enp0s9 is up again but enp0s10 stays active: with no primary port set, active-backup doesn’t switch back, which avoids a second interruption. If you want a preferred port, add primary=enp0s9 to bond.options. The failure count of 1 stays until reboot, which is how you spot a flapping cable during an audit.
✅ Checkpoint: Active port moved to enp0s10 when NIC3 was unplugged, and enp0s9 shows MII Status up with Link Failure Count 1 after you plugged it back.
Break It, Then Fix It
Take a snapshot of rocky01 first (VBoxManage snapshot rocky01 take before-l31 on the laptop). The first trap bites only after a reboot, the second locks you out over SSH on purpose, and the third does nothing at all, which is the problem.
1A route added with ip route add vanishes after a reboot
Add a route the quick way and check it:
sudo ip route add 10.60.0.0/16 via 10.10.10.1
ip route show 10.60.0.0/16
sudo reboot
# after logging in again:
ip route show 10.60.0.0/16; echo "lines: $(ip route show 10.60.0.0/16 | wc -l)"You’ll get this:
10.60.0.0/16 via 10.10.10.1 dev enp0s3
lines: 0Before the reboot the route was there and traffic would have worked, so whoever added it during an incident closed the ticket happily. It was only in the kernel’s memory, never in a keyfile, so after the reboot it is gone, while your 10.50.0.0/16 route from the lab is still there. Note also there is no “proto static” on it: that tag means NetworkManager added it.
Fix: add it to the connection. Watch the syntax, because the ip habit of writing “via” doesn’t work here:
sudo nmcli connection modify enp0s3 +ipv4.routes "10.60.0.0/16 via 10.10.10.1"You’ll get this error:
Error: failed to modify ipv4.routes: The valid syntax is: 'ip[/prefix] [next-hop] [metric] [attribute=val]... [,ip[/prefix] ...]'.sudo nmcli connection modify enp0s3 +ipv4.routes "10.60.0.0/16 10.10.10.1"
sudo nmcli device reapply enp0s3
nmcli -g ipv4.routes connection show enp0s3You should see something like this:
Connection successfully reapplied to device 'enp0s3'.
10.50.0.0/16 10.10.10.1, 10.60.0.0/16 10.10.10.1Both routes are in the profile now. 10.60.0.0/16 was only for this exercise, so remove it with the minus sign and reapply:
sudo nmcli connection modify enp0s3 -ipv4.routes "10.60.0.0/16 10.10.10.1"
sudo nmcli device reapply enp0s3✅ Checkpoint: nmcli -g ipv4.routes connection show enp0s3 prints only 10.50.0.0/16 10.10.10.1.
2Change the IP over SSH and lock yourself out
From the laptop, SSH in through the port forward (ssh -p 2211 student@127.0.0.1) and run:
sudo nmcli connection modify enp0s3 ipv4.addresses 10.10.10.12/24
sudo nmcli connection up enp0s3Your SSH prompt stops responding after the second command and never comes back. The port forward on the LinuxLab network still points at 10.10.10.11, and rocky01 doesn’t have that address any more. A new connection from the laptop fails too:
ssh -p 2211 student@127.0.0.1You’ll get something like this (the wording depends on your SSH client):
kex_exchange_identification: Connection closed by remote host
Connection closed by 127.0.0.1 port 2211On a server in a data centre this is the moment you call someone to stand in front of it, or open the out-of-band console. Here, the VirtualBox console window is your out-of-band console.
Fix: on the rocky01 console, put the address back.
sudo nmcli connection modify enp0s3 ipv4.addresses 10.10.10.11/24
sudo nmcli connection up enp0s3
ip -br address show enp0s3When you have to change a remote server’s address and can’t use the console, prepare an automatic undo first. Copy the keyfile, schedule a restore in five minutes with a transient systemd timer (Lesson 28), make the change, and cancel the timer only once you have logged in on the new address:
sudo cp -p /etc/NetworkManager/system-connections/enp0s3.nmconnection /root/enp0s3.nmconnection.good
sudo systemd-run --on-active=5min --unit=net-undo /bin/sh -c 'cp -p /root/enp0s3.nmconnection.good /etc/NetworkManager/system-connections/enp0s3.nmconnection && nmcli connection reload && nmcli connection up enp0s3'
# make the change; if you can log in on the new address:
sudo systemctl stop net-undo.timer✅ Checkpoint: You can SSH to rocky01 through port 2211 again, and enp0s3 shows 10.10.10.11/24.
3Modify without applying
Change the DNS server in the profile and look at what the system uses:
sudo nmcli connection modify enp0s3 ipv4.dns "10.10.10.53"
nmcli -g ipv4.dns connection show enp0s3
grep nameserver /etc/resolv.confYou’ll get this:
10.10.10.53
nameserver 10.10.10.1No error and no change on the system. The profile says 10.10.10.53, but resolv.conf and the device still use 10.10.10.1, and they will until the connection is applied, or until the next reboot, when the change you forgot about suddenly takes effect. This is how a server “breaks by itself” a month after someone’s change.
Fix: put the right value back and apply it, then confirm both sides agree.
sudo nmcli connection modify enp0s3 ipv4.dns "10.10.10.1"
sudo nmcli device reapply enp0s3
nmcli -g ipv4.dns connection show enp0s3; nmcli -g IP4.DNS device show enp0s310.10.10.1
10.10.10.1✅ Checkpoint: The connection and the device both report DNS 10.10.10.1.
This happened at work
Four hundred agents and no dial tone
I was on the Linux team for a BPO call centre in Pune when this happened, on the Sunday night after our monthly kernel patching. The dialer servers, which connect agents’ calls to the telecom provider’s voice network, rebooted cleanly at 2 a.m. and every check on the patching sheet was green. At 7 a.m. the morning shift of about four hundred agents logged in and not a single outbound call connected.
The application team restarted the dialer service twice, then the whole server, because the dialer logs only said “SIP trunk unreachable”. When I got on, ip route get for the provider’s signalling address showed the traffic going out of the default gateway towards the internet firewall, which dropped it. The provider’s network, 10.50.0.0/16 in our case too, was supposed to go through a separate router on the voice VLAN.
That route had never been in the server’s configuration. During go-live a year earlier, an engineer had added it with ip route add to get the first calls working, it stayed in memory through eleven months without a reboot, and the patching reboot was the first one since. I added it properly with nmcli connection modify ... +ipv4.routes and a device reapply, calls started within a minute, and we had lost about forty minutes of a Monday morning shift.
We changed three things after that. Every network change ticket now needs the nmcli command in it, not the ip command. The health check script after patching runs ip route get for each critical destination and compares the gateway with the expected one.
And new servers get a test reboot before go-live, so anything living only in memory shows up while nobody is waiting on it.
Common mistakes and troubleshooting
| Mistake | What happens | What to do instead |
|---|---|---|
Adding routes or addresses with ip for a permanent change | Gone after the next reboot or reconnect | Use nmcli connection modify with +ipv4.routes or +ipv4.addresses |
| Running modify and walking away | The profile changes, the system doesn’t, until a reboot surprises you | Follow with nmcli device reapply DEV or nmcli connection up NAME, then check both sides |
Writing "10.50.0.0/16 via 10.10.10.1" in nmcli | Error: failed to modify ipv4.routes | Destination, space, next hop: "10.50.0.0/16 10.10.10.1" |
| Following a guide that edits ifcfg files or uses teamd on RHEL 10 | Files are ignored and team connections are unsupported | Use keyfiles through nmcli and kernel bonding; run nmcli connection migrate on RHEL 9 before upgrading |
| Editing a keyfile by hand and leaving it at the wrong mode | NetworkManager doesn’t load it, or keeps the old settings | Keep it root:root 600, then nmcli connection reload and connection up |
| Changing the address of the interface you are logged in through | The session hangs and you need someone at the console | Use the console, or schedule an automatic undo with systemd-run first |
| Cabling both bond ports to the same switch | A switch reboot takes the server off the network anyway | Ports to two switches; check with /proc/net/bonding/bond0 during switch maintenance |
Interview questions
Click a question to open it. Each one is written the way an interviewer would ask it. Under it you will see why they ask it and one way you could answer, in your own words.
Fresher level
1. In nmcli, what’s the difference between a device and a connection?
Why they ask this: A first check that you understand NetworkManager’s model, because every other nmcli command depends on it.
How you might answer: A device is an interface the kernel has, like enp0s3 or bond0. A connection is a saved profile of settings, stored as a keyfile in /etc/NetworkManager/system-connections, that gets activated on a device.
So nmcli connection modify enp0s3 changes the profile, and nmcli device show enp0s3 shows what is applied right now. On many servers both have the same name, which is why people mix them up.
2. How do you set a static IP on a RHEL 10 server so it survives a reboot?
Why they ask this: They want the exact command and to hear that you apply and verify it.
How you might answer: I use nmcli connection modify on the connection with ipv4.method manual, ipv4.addresses with the prefix, ipv4.gateway and ipv4.dns, for example 10.10.10.11/24 with gateway 10.10.10.1. That writes the keyfile.
Then nmcli connection up to apply it, from the console if it’s the interface I’m connected through, and I check with ip -br addr and ip route. On RHEL 10 there are no ifcfg files, so I don’t look for them.
2–5 years experience
3. During an incident a colleague fixed connectivity with ip route add. What do you do once things are stable?
Why they ask this: This checks whether you think about what happens at the next reboot, not only the current fix.
How you might answer: I’d make the route permanent in NetworkManager with nmcli connection modify CON +ipv4.routes "10.50.0.0/16 10.10.10.1" and a device reapply, then confirm nmcli -g ipv4.routes and ip route agree.
I’d also record it in the change ticket and our config management, and if possible test a reboot in a quiet window, because a route that lives only in memory is a planned outage at the next patch night.
4. Which bonding mode would you choose for a database server, and why?
Why they ask this: They’re checking that you know the main modes and what each needs from the network team.
How you might answer: If the switches support it and the network team can configure LACP across two switches (MLAG or vPC), I’d use 802.3ad, mode 4, to use both links. Otherwise active-backup, mode 1, which needs no switch config and fails over in about 100 ms with miimon=100.
For a database the main goal is surviving a switch or cable failure, so either way the two ports go to different switches. I’d verify with /proc/net/bonding/bond0 and pull a cable during a test window to prove it works.
Senior level
5. You’re planning the upgrade of 200 RHEL 9 servers to RHEL 10. What networking issues do you plan for?
Why they ask this: Seniors should know the removed features and how to find affected servers before the upgrade, not during it.
How you might answer: First I’d inventory which servers still have ifcfg profiles, teams, or scripts calling ifup, ifdown or dhclient, because RHEL 10 removed all of those. Ansible can collect nmcli -g NAME,TYPE,FILENAME connection show from every server.
Then on RHEL 9 I’d run nmcli connection migrate to convert ifcfg profiles to keyfiles, rebuild teams as bonds with the same IP, and fix the scripts, all before Leapp. I’d also look at monitoring that parses ifcfg files and update our build templates so new RHEL 9 servers don’t add to the list.
6. How would you change the IP address of a remote production server safely?
Why they ask this: They want to hear about rollback, console access and coordination with DNS and firewalls.
How you might answer: I’d confirm out-of-band console access works before the window, and get firewall rules, DNS records and monitoring ready for the new address. Where possible I add the new address as a second address first, test it, move DNS, and only remove the old one later.
If I have to switch in one step, I schedule an automatic undo with a transient systemd timer that restores the saved keyfile and reactivates the connection, and cancel it only after I log in on the new IP. nmstate’s apply with a checkpoint does the same rollback for teams that use it.
Quick quiz
Q1. Where does RHEL 10 store the settings for the enp0s3 connection?
/etc/sysconfig/network-scripts/ifcfg-enp0s3/etc/NetworkManager/system-connections/enp0s3.nmconnection/etc/netplan/enp0s3.yaml/etc/network/interfaces
Show answer
B — RHEL 10 removed ifcfg support; profiles are keyfiles in system-connections.
Q2. You ran nmcli connection modify enp0s3 ipv4.dns 10.10.10.53 and nothing changed. Why?
- The DNS server is wrong
- modify only changes the profile; it needs connection up or device reapply
- nmcli needs a reboot for DNS
- DNS can only be set in resolv.conf
Show answer
B — modify writes the keyfile. Apply it with nmcli device reapply enp0s3 or nmcli connection up enp0s3.
Q3. Which command adds a persistent route without removing existing ones?
nmcli connection modify enp0s3 ipv4.routes "10.50.0.0/16 10.10.10.1"nmcli connection modify enp0s3 +ipv4.routes "10.50.0.0/16 10.10.10.1"ip route add 10.50.0.0/16 via 10.10.10.1nmcli connection modify enp0s3 +ipv4.routes "10.50.0.0/16 via 10.10.10.1"
Show answer
B — The plus sign appends to the list; no sign replaces it; ip is temporary; “via” is rejected.
Q4. In an active-backup bond, the active port loses its link. What happens?
- The bond goes down until the link returns
- The backup port becomes active and the bond keeps its IP and MAC
- Both ports share traffic
- The switch must be reconfigured
Show answer
B — The kernel detects the failure (miimon), moves the bond to the backup port and announces the MAC with gratuitous ARP.
Q5. What should you use on RHEL 10 instead of network teaming (teamd)?
- firewalld zones
- A bridge
- Kernel bonding with NetworkManager
- dhclient
Show answer
C — teamd and libteam were removed in RHEL 10; Red Hat recommends bonds.
Quick recap
- nmcli static IP:
nmcli connection modify CON ipv4.method manual ipv4.addresses IP/PREFIX ipv4.gateway GW ipv4.dns DNS, thenconnection upordevice reapply. - A device is the interface; a connection is the saved profile, a keyfile in /etc/NetworkManager/system-connections (root, mode 600). RHEL 10 has no ifcfg files.
- Persistent routes:
+ipv4.routes "10.50.0.0/16 10.10.10.1"(no “via”).ip route addis lost at reboot. - Use + and – to add or remove list items; read single values with
nmcli -g. - Bonds: bond0 holds the IP and
bond.options, ports useport-type bond controller bond0; check/proc/net/bonding/bond0. teamd is gone in RHEL 10. - Change the interface you are connected through only from a console, or with an automatic undo scheduled.
Next lesson
Lesson 32: Static IP and routes with netplan on Ubuntu → You’ll do the same jobs on ubu01 the Ubuntu way: a netplan YAML file for the address, DNS and routes, tested with netplan try so a mistake rolls itself back.
Further reading: nmcli static IP
Want to go deeper into nmcli static IP? Read Red Hat’s networking changes in RHEL 10 (ifcfg, teamd and dhclient removals), the NetworkManager reference for nmcli and the keyfile format, and the Linux kernel’s bonding driver documentation. You can also see the full plan for this course on the undefined course page.