Upgrading my server to AlmaLinux 9

I have been running my current server running CentOS 7 for almost 8 years. It runs Plex, acts as a NAS, and runs a few other small things. CentOS 7 is almost at the end of its lifecycle, and, with Red Hat recently cancelling CentOS as we had always known it, it was time to find a replacement.

Fortunately both Rocky Linux and AlmaLinux stepped up to fill that gap. While Red Hat Enterprise Linux (RHEL) is open source, it’s not free. Rocky and Alma are essentially free RHEL clones. While I initially expected Rocky to be the more successful of the two, AlmaLinux was first off the blocks, releasing AlmaLinux 9.0 on 2022-05-26, just 9 days after RHEL 9.0’s launch. That’s simply incredible, and they’ve likely won the battle with Rocky already. As of mid-July, Rocky Linux 9 is still not out [Edit. Rocky Linux 9 was released on 14/July, the day after I installed Alma]. As both Alma and Rocky provide an identical product, speed of releases is the only differentiator. AlmaLinux clearly made a big investment to be first, and I’m pretty confident it will pay off.

RHEL has never really supported in-place upgrades, and I don’t blame them. GNU/Linux changes a lot over the years. Just imagine building support for an in-place upgrade when the OS switches from sysvinit to upstart, or upstart to systemd. Or has to migrate from iptables to firewalld, or move from X.org server to Wayland. No thanks.

Installing AlmaLinux 9 meant a fresh install. That’s fine actually, as an 8-year-old install is going to have some cruft. I also wanted an easy way to undo the install if something went wrong, so I bought a cheap $25 SATA SSD (256GB) and installed it there. I will keep my old SSD with my CentOS 7 install for a couple of days should I need to switch back.

I run the server headless (no GUI), so besides just wanting the latest and greatest, or simply wanting to run a supported system, did I even notice anything after upgrading? Well, a few things:

• firewalld is much improved. I can now easily poke a hole in the firewall by service, instead of by port. That means a single command to allow remote Plex streaming, and only that. Don’t get me started with how awful SELinux was back in the RHEL7 days. Everyone just disabled it because it broke everything.
• exFat is now officially supported. It’s the ideal filesystem for USB drives where you need multi-OS compatibility. Windows, Mac, and now Linux can all read it without any extra drivers or hacks.
• Cockpit is shaping up nicely. I don’t really use it, but it does seem interesting.
• Automatic updates are fully supported (you can enable all updates, or just security updates to be installed automatically).
• Kernel live patching is supported. I generally don’t have super high uptime requirements, so this isn’t a huge deal, but it’s pretty cool that this is now free and easy to enable. Both Kernel live patching and automatic updates can be enabled with one click through Cockpit. I’m not sure how comfortable I am with this. I generally like to reboot immediately after a kernel update. That way if it breaks I know what to blame. If I get silent kernel updates, and months later I reboot and it doesn’t come back up, I’ll not know the root cause, and likely won’t be able to roll back.

I’ve had my AL9 server running for a few days now, and everything works as expected. It looks like Red Hat abandoning everyone by cancelling CentOS turned out to be no big deal! The OSS community came through!

So, why do I still use a RHEL-clone instead of Debian/Ubuntu/Arch?

Debian has 3 years of guaranteed updates (though it sometimes gets more), and Ubuntu LTS releases get 5 years. I don’t want to be forced to upgrade. I want to choose to upgrade. In my case I skipped CentOS 8 because I didn’t have the time to upgrade. That turned out to be a lucky decision, as CentOS 8 was later cancelled by Red Hat mid lifecycle. I was able to calmly wait for a RHEL9 clone before upgrading.

And as for why not Arch Linux? I feel Arch is an excellent desktop OS. In fact, I’ve been using Arch on my desktop since 2008. But I don’t consider it a good server OS. On the desktop I want the latest video drivers to play games on Steam. I want the latest desktop GUI. Most of all, I want AUR, the Arch User Repository, where almost every app you could want is a simple install away. In fact, when I built my desktop in 2020, only the very latest kernel supported my Wi-Fi chip, and only an rc kernel supported my video card. On Arch that was all pretty easy to sort out. On another OS? No.

But for my server, I don’t need any of that. I need stability. Both in the ‘don’t crash’ sense, and the ‘don’t change stuff’ sense. Do I want every single kernel dot release. No. Do I want an in-place upgrade from X.org to Wayland with no warning. How about an upgrade PulseAudio, or a messy upgrade from Python 2 to Python 3? No. I want to set my server up and forget it. For my server needs, I don’t think there’s anything better than RHEL.

Also, RHEL is the server OS I’m most familiar with. My university labs ran a combination of CentOS 3 and Fedora Core 1. Then I worked at HP, where everything was RHEL 5. Then I went to work at Ancestry.com where we ran CentOS 6. I’m now at AWS where we use Amazon Linux. Amazon Linux used to be a RHEL fork, but recent versions are now built from Fedora Linux.

[update] Rocky Linux 9 was released on 14/July, just one day after I installed AlmaLinux9 on my server. So, do I regret not having waited? Not really. On the one hand, I did expect Rocky to be the generally preferred option due to not being connected to a commercial OS (Alma is made by the company behind CloudLinux OS). But on the other hand, it was nice not to have a choice. I mean they’re essentially identical OSes, so speed is the main differentiator, and Alma are currently thrashing Rocky. Still, if something changes, I imagine it would be pretty trivial to switch between them. I’ll be most interested in seeing if Alma maintains their lead, or if Rocky catches up. The main upside though, is that both Alma and Rocky released v9 much faster than CentOS did with v8. Maybe Red Hat dumping CentOS will turn out to be a good thing!