Major OS upgrades: Leapp, do-release-upgrade and a rollback plan

Part A, steps 1 to 6, runs on rocky01 and ubu01 and changes nothing except a baseline folder in /root and a VirtualBox snapshot. Part B, steps 7 to 9, is optional and runs on a new throwaway VM called ubu24 that you build from the Ubuntu Server 24.04 ISO and delete at the end. Step 10 is optional too and needs a Red Hat Developer account.

On an 8 GB laptop, shut rocky01 down before you start ubu24.

1Write down what you run, to the minor version

Run this on both VMs:

grep -E '^(NAME|VERSION)=' /etc/os-release
uname -r

You should see this on rocky01:

NAME="Rocky Linux"
VERSION="10.2 (Red Quartz)"
6.12.0-211.16.1.el10_2.0.1.x86_64

And on ubu01:

NAME="Ubuntu"
VERSION="26.04.1 LTS (Resolute Raccoon)"
7.0.0-15-generic

The supported-paths table works on exact minor versions, so “RHEL 9” isn’t enough; you need “9.8”. Paste these lines into the change ticket before anything else.

✅ Checkpoint: You have the exact version and kernel of both VMs written down.

2Ask each system what upgrade it is offered

On ubu01:

grep -v '^#' /etc/update-manager/release-upgrades | grep .
sudo do-release-upgrade -c

You should see this:

[DEFAULT]
Prompt=lts
Checking for a new Ubuntu release
There is no development version of an LTS available.
To upgrade to the latest non-LTS development release
set Prompt=normal in /etc/update-manager/release-upgrades.

Prompt=lts means “offer me only the next LTS”, and 26.04 is the newest LTS, so there is nothing to offer. The message about a non-LTS release refers to 26.10, a nine-month release you wouldn’t put on a server. On a 24.04 server today, the same command offers 26.04.1.

On rocky01:

cat /etc/rocky-release
dnf repolist

You should see something like this:

Rocky Linux release 10.2 (Red Quartz)
repo id                       repo name
appstream                     Rocky Linux 10 - AppStream
baseos                        Rocky Linux 10 - BaseOS
extras                        Rocky Linux 10 - Extras

Rocky’s repositories are for “10”, so 10.2 to 10.3 arrives as a normal dnf update (Lesson 35). There is no command that offers Rocky 11 or takes a Rocky 9 server to 10; that path is a rebuild.

✅ Checkpoint: ubu01 says no LTS is available beyond 26.04, and you can explain why rocky01 has no major upgrade command.

3Check free space where the upgrade needs it

Run this on both VMs:

df -h / /boot /var

You should see something like this on rocky01:

Filesystem           Size  Used Avail Use% Mounted on
/dev/mapper/rl-root   17G  3.1G   14G  19% /
/dev/sda2            1.9G  418M  1.5G  22% /boot
/dev/mapper/rl-root   17G  3.1G   14G  19% /

And on ubu01:

Filesystem                         Size  Used Avail Use% Mounted on
/dev/mapper/ubuntu--vg-ubuntu--lv  8.8G  4.6G  3.8G  55% /
/dev/sda2                          2.0G  198M  1.6G  11% /boot
/dev/mapper/ubuntu--vg-ubuntu--lv  8.8G  4.6G  3.8G  55% /

/var appears as / because it isn’t a separate file system on these VMs. An upgrade downloads several hundred MB to 2 GB of packages into /var and puts a new kernel and initramfs in /boot. A /boot that’s more than half full is the classic reason an upgrade stops halfway; remove old kernels first (Lesson 49).

✅ Checkpoint: Each VM has at least 3 GB free on / and more than 300 MB free on /boot.

4Find packages and repositories that aren’t from your distribution

On ubu01:

ls /etc/apt/sources.list.d/
grep -h '^URIs' /etc/apt/sources.list.d/*.sources
apt-mark showhold
apt list --installed 2>/dev/null | grep -c ',local]'

You should see something like this:

ubuntu.sources
URIs: http://archive.ubuntu.com/ubuntu/
URIs: http://security.ubuntu.com/ubuntu/
0

Only Ubuntu’s own archive and security mirrors, no held packages (showhold printed nothing) and zero packages marked local, which is how apt labels an installed package that no configured repository provides. Any other URI is a third-party repository you must check with its vendor before the upgrade.

On rocky01:

sudo dnf repoquery --installed --qf '%{from_repo}' | sort | uniq -c

You should see something like this:

    318 anaconda
     46 appstream
     21 baseos

Each installed package is counted by the repository it came from; anaconda means “installed from the ISO”. Anything else, such as epel or a vendor repo, needs a version for the target release before you upgrade. At work also run dnf versionlock list if the versionlock plugin is installed, because Red Hat’s preparation steps ask you to clear locks first.

✅ Checkpoint: You can list every repository each VM uses and confirm there are no held or locked packages.

5Save a baseline you can compare against later

Paste this whole block on both VMs:

sudo tee /usr/local/sbin/preupgrade-baseline.sh >/dev/null <<'EOF'
#!/bin/bash
# preupgrade-baseline.sh: save what this server looks like before a major upgrade (Lesson 52)
# Run it as root: once before the upgrade, once after, then compare the two folders with diff.
set -u
out="/root/baseline-$(hostname -s)-$(date +%Y%m%d-%H%M)"
mkdir -p "$out" && cd "$out" || exit 1

cat /etc/os-release > os-release.txt
uname -r > kernel.txt
systemctl list-units --type=service --state=running --no-legend --plain > services-running.txt 2>&1
systemctl list-unit-files --type=service --state=enabled --no-legend > services-enabled.txt 2>&1
systemctl --failed --no-legend --plain > services-failed.txt 2>&1
ss -tulpn > ports.txt 2>&1
findmnt --real > mounts.txt 2>&1
df -hT > df.txt 2>&1
ip -br addr > ip-addr.txt 2>&1
ip route > ip-route.txt 2>&1
if [ -f /etc/debian_version ]; then
  dpkg-query -W -f '${Package} ${Version}\n' | sort > packages.txt
else
  rpm -qa | sort > packages.txt
fi
crontab -l > root-crontab.txt 2>&1
tar -czf etc.tar.gz -C / etc 2>/dev/null

echo "Baseline saved in $out"
ls "$out"
EOF
sudo chmod 755 /usr/local/sbin/preupgrade-baseline.sh
sudo /usr/local/sbin/preupgrade-baseline.sh

You should see something like this on rocky01:

Baseline saved in /root/baseline-rocky01-20261027-1142
df.txt
etc.tar.gz
ip-addr.txt
ip-route.txt
kernel.txt
mounts.txt
os-release.txt
packages.txt
ports.txt
root-crontab.txt
services-enabled.txt
services-failed.txt
services-running.txt

Thirteen small files that answer “what did this server look like before?”: running and enabled services, failed units, listening ports, mounts, disk use, addresses, routes, every package with its version, root’s crontab and a copy of /etc. Copy the folder off the server too (scp to the laptop), because a rollback by snapshot restores the old folder with everything else.

✅ Checkpoint: /root/baseline-rocky01-DATE and /root/baseline-ubu01-DATE exist with 13 files each.

6Take a snapshot and write the rollback plan

On the laptop (PowerShell or a terminal):

VBoxManage snapshot ubu01 take before-upgrade --description "Lesson 52 rollback point"
VBoxManage snapshot ubu01 list

You should see something like this:

0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Snapshot taken. UUID: 6c1d2e8a-4f3b-4a7e-9b0c-2d5e8f1a7c34
   Name: base (UUID: 0a9b3c4d-1e2f-4a5b-8c6d-7e8f9a0b1c2d)
      Name: before-upgrade (UUID: 6c1d2e8a-4f3b-4a7e-9b0c-2d5e8f1a7c34) *

The star marks the current state. A snapshot of a running VM takes a few seconds; for databases at work, stop the application first or use the backup tool’s own method (Lesson 63), so the snapshot holds consistent data.

Now copy this template into a file and fill in the blanks for ubu01:

Upgrade and rollback plan
Server:            ubu01 (10.10.10.21), lab NFS and iSCSI server
From / to:         Ubuntu 26.04.1 LTS -> next LTS (practice plan)
Approved slot:     Sat 02:00-05:00 IST, change ticket ________
Backup:            ID ________, restore tested on ________
Snapshot:          VirtualBox "before-upgrade", taken at ________
Baseline:          /root/baseline-ubu01-<date>, copied to the laptop
Go/no-go checks:   space OK, no held packages, no pending reboot,
                   no third-party repos without a new-version build,
                   vendor supports the new version
Rollback trigger:  any application check fails, or not healthy by 04:00
Rollback steps:    power off, restore snapshot "before-upgrade",
                   start, run the health check, tell the app owner
After success:     keep the snapshot 3 days, then delete it
Contacts:          app owner ________, DBA ________, network ________

The trigger and the deadline are the most important lines. Decide them while you’re calm, so that at 04:00 the call is “the plan says roll back” rather than “one more try”. Keep the snapshot until Lesson 53, then delete it.

✅ Checkpoint: ubu01 has a snapshot called before-upgrade and you have a filled-in plan with a rollback trigger and deadline.

7Optional: build the throwaway Ubuntu 24.04 VM

In VirtualBox create a VM called ubu24 (2 vCPU, 2048 MB, 20 GB disk, NIC on the LinuxLab NAT Network) from the Ubuntu Server 24.04 ISO. In the installer give it 10.10.10.29/24, gateway and DNS 10.10.10.1, user student, and tick OpenSSH server. When it boots, take a snapshot before you update it, update it, then take another:

On the laptop, then inside ubu24, then on the laptop again:

VBoxManage snapshot ubu24 take fresh-install

sudo apt update && sudo apt full-upgrade -y
[ -f /var/run/reboot-required ] && sudo reboot

VBoxManage snapshot ubu24 take updated

The fresh-install snapshot is for the Break-It part. Lesson 33’s SSH hardening isn’t on this VM, so password logins over SSH still work; that’s fine for a VM that lives for one evening.

[Add your screenshot here]: the Ubuntu Server 24.04 installer network screen with 10.10.10.29/24 set on the NIC.

✅ Checkpoint: lsb_release -ds on ubu24 shows Ubuntu 24.04 with the latest point release, and both snapshots exist.

8Optional: run do-release-upgrade over SSH, the safe way

From ubu01, copy the baseline script across, save a baseline and check the offer:

scp /usr/local/sbin/preupgrade-baseline.sh student@10.10.10.29:
ssh student@10.10.10.29
sudo install -m 755 preupgrade-baseline.sh /usr/local/sbin/
sudo preupgrade-baseline.sh
sudo do-release-upgrade -c

The last command should print:

Checking for a new Ubuntu release
New release '26.04.1 LTS' available.
Run 'do-release-upgrade' to upgrade to it.

Start the upgrade:

sudo do-release-upgrade

Because you’re on SSH, it asks this first (answer y):

Continue running under SSH?

This session appears to be running under ssh. It is not recommended
to perform a upgrade over ssh currently because in case of failure it
is harder to recover.

If you continue, an additional ssh daemon will be started at port
'1022'.
Do you want to continue?

Continue [yN]

The spare sshd on port 1022 is your second door if the main one breaks during the upgrade. ubu24 has no firewall enabled, but on a server where ufw is on, as on ubu01 since Lesson 34, open it first with sudo ufw allow 1022/tcp and remove the rule afterwards. If your SSH session drops, log in again and run sudo do-release-upgrade once more: it finds the running GNU screen session and reattaches to it.

After it calculates the changes you’ll see something like this; read it, then answer y:

Do you want to start the upgrade?

6 installed packages are no longer supported by Canonical. You can
still get support from the community.

12 packages are going to be removed. 148 new packages are going to be
installed. 702 packages are going to be upgraded.

You have to download a total of 812 M. This download will take about
2 minutes with your connection.

Installing the upgrade can take several hours. Once the download has
finished, the process cannot be canceled.

 Continue [yN]  Details [d]

Press d to see the package list if anything looks odd. Later it asks whether to keep or replace config files you changed (keep yours unless you know why the new one is better), whether to remove obsolete packages, and finally to restart. The whole run takes 20 to 60 minutes on a laptop.

✅ Checkpoint: ubu24 restarts on its own at the end of the upgrade.

9Optional: verify against the baseline, then delete the VM

Log in to ubu24 again and run:

lsb_release -ds
uname -r
systemctl --failed --no-legend | wc -l
sudo preupgrade-baseline.sh >/dev/null
sudo sh -c 'diff /root/baseline-ubu24-*/services-running.txt'
grep -l '^Enabled: no' /etc/apt/sources.list.d/*.sources

You should see something like this for the first three commands:

Ubuntu 26.04.1 LTS
7.0.0-15-generic
0

The glob matches the before and after folders, so diff compares the running services: lines starting with < ran before and don't now. On a fresh VM the list is short; on a real server every line is a question for the application owner. The last grep lists sources the upgrader switched off, which is none here and is where you'd look for a vendor's repo on a real server.

When you’re done, on the laptop:

VBoxManage controlvm ubu24 poweroff
VBoxManage unregistervm ubu24 --delete

✅ Checkpoint: ubu24 reported 26.04.1 with zero failed units, and it no longer appears in VirtualBox.

10Optional: a Leapp pre-upgrade report on RHEL 9.8

This needs a no-cost Red Hat Developer subscription and a throwaway RHEL 9.8 VM (call it rhel98, 10.10.10.28, 2 GB RAM). The commands follow Red Hat’s RHEL 9 to 10 guide; check the current guide for the leapp package versions it expects.

On rhel98, as root or with sudo:

sudo subscription-manager register
sudo subscription-manager repos --enable rhel-9-for-x86_64-baseos-rpms --enable rhel-9-for-x86_64-appstream-rpms
sudo subscription-manager release --set 9.8
sudo dnf install -y leapp-upgrade
sudo dnf update -y && sudo reboot
sudo leapp preupgrade --target 10.2

You should see something like this at the end:

============================================================
                      REPORT OVERVIEW
============================================================

Reports summary:
    Errors:                      0
    Inhibitors:                  0
    HIGH severity reports:       1
    MEDIUM severity reports:     1
    LOW severity reports:        3
    INFO severity reports:       4

Before continuing, review the full report below for details about discovered problems and possible remediation instructions:
    A report has been generated at /var/log/leapp/leapp-report.txt
    A report has been generated at /var/log/leapp/leapp-report.json

============================================================
                   END OF REPORT OVERVIEW
============================================================

Answerfile has been generated at /var/log/leapp/answerfile

Zero inhibitors means Leapp would let you continue. Read every high and medium finding in leapp-report.txt anyway, because each one comes with a remediation hint, and check /var/log/leapp/answerfile for questions. Stop here; leapp upgrade --target 10.2 is the step that changes the VM, and it’s yours to try on a throwaway VM with a snapshot.

✅ Checkpoint: leapp-report.txt exists and you can name the risk level of each finding.