Cannot boot after CentOS 8.1 update

[advsp]

I had a server with GUI system running CentOS 8.0 OK. On Friday software update prompted me to update so I did. I did not realize the list of updates included a full update to 8.1. After the update and requested restart it would not boot into regular mode. It goes to dracut and says my /boot and /swap are not accessible. I CAN boot into recovery mode and all partitions are found and accessible.

This is a 10-series Dell server so I am using the MegaRAID_SAS driver from elrepo via DUD for a RAID 5 array that contains /root, /swap, /home. The MegaRAID driver for v8.0 was successfully installed and working before. It did not automatically update the driver to the v8.1 version, which I know is required, so I booted into recovery and added it with the rpm command, no problem with the install there. Still no boot to v8.1. Hmmm.

Since there were now 2 versions of the driver installed I thought it might be confusing the system so I removed the v8.0 version using rpm -e. Still no boot to v 8.1.

So I removed the v8.1 driver to add back the v8.0 driver and then tried to boot into v8.0 by selecting it from the list of choices at the boot menu. No boot to v8.0 now either. But I can still boot to recovery mode.

Any thoughts on what I should do? Are there any reports of the upgrade from 8.0 to 8.1 breaking installations? From what I have found on the Internet it does not seem like it is the MegaRAID_SAS v8.1 driver itself as many report it to be working fine. Plus, it is uninstalled now. What in the 8.1 update could have broken the 8.0 boot?

[toracat]

So, you were able to install 8.0 using ELRepo's DUD (for 8.0), and now you cannot run 8.1 even if you use ELRepo's driver (for 8.1) ? As you noted, kmods built for 8.0 need to be updated to those built for 8.1.

[advsp]

Correct, 8.0 with 8.0 DUD worked and now after the upgrade to 8.1 and adding the 8.1 DUD it will not boot to neither 8.1 nor 8.0.

[toracat]

You'd need a kmod package, not DUD once updated to 8.1. Assuming your hardware uses the kmod-megaraid_sas package, you need to install kmod-megaraid_sas-07.707.51.00-1.el8_1.elrepo.x86_64.rpm.

If elrepo is enabled during the 8.0 to 8.1 update, that kmod package should be updated automatically.

[advsp]

Yes, that is the package that I installed. ELRepo was not enabled during the update, nor still. I guess that would have been much more convenient but nothing I was reading ever mentioned enabling the repo, just "download this file". Should I enable now via the recovery environment? Would that help?

I installed from an 8.1 kmod DUD that I made similar to the kmod DUD I made for the 8.0 driver when I first installed. That was the suggested method for initial installation on the ELRepo blog (and others) and was a convenient way to get the 8.1 driver update to the server too.

I used rpm -ivh /path/to/DUD/RPMS/x86_64/kmod-megaraid_sas-07.707.51.00-1.el8_1.elrepo.x86_64. I did NOT invoke Anaconda for the update, of course. That install seemed to go just fine and the package showed as installed with rpm -qa *megaraid*.

One thing I was wondering was if that kmod must only be installed via Anaconda along with an installation of CentOS and if some related configuration had not been completed because I only used rpm to install???

Also, I am not 100% sure that the reason it isn't booting is the RAID driver. Seemed likely since it tells me my partitions are not available but that was an assumption. Should I be confirming the cause first? If so, how?

[toracat]

Installing the elrepo repo will help in future system updates. Once installed, it is enabled by default.

You don't need to install the kmod package through anaconda. Installing using the rpm file is fine. However, it must be updated at the same time the OS update. In this case the kmod built for the 8.0 kernel is not compatible with the 8.1 kernel. So the kmod built against the 8.1 kernel needs to be installed before booting 8.1.

I realize the ELRepo blog is not quite up to date. Will try improving it.

[advsp]

OK, so, as I wrote, I installed the 8.1 driver after the first boot attempt with 8.1 and ti still would not boot after that. I reinstalled 8.1 driver a little while ago and it still is not booting with 8.1 nor 8.0. Is there any further configuration that needs to be done? Could there be another cause for it not booting? What should I try next?

[toracat]

Do you see any messages/errors while booting?

[advsp]

No, not during the boot attempt. At the end when it cannot boot it goes into emergency mode and says that my /root and /swap are not found. Are there any logs that would give a clue? Any other ideas?

[toracat]

I have another idea. As a CentOS user, you can use the centosplus kernel. It is here:

http://mirror.centos.org/centos/8/cento ... /Packages/

The plus kernel has a number of drivers that were removed from RHEL-8 kernel (including megaraid_sas).