kernel-4.18.0-240.22.1.el8_3.x86_64 installation trouble?

Issues related to applications and software problems and general support
Post Reply
donsizemore
Posts: 4
Joined: 2021/04/09 19:51:26

kernel-4.18.0-240.22.1.el8_3.x86_64 installation trouble?

Post by donsizemore » 2021/04/09 19:57:43

Just seeing who has upgraded to 4.18.0-240.22.1.el8_3.x86_64 so far.

On three of my CentOS 8.3 systems, in XCP-NG, the new kernel installed cleanly and came up.

On the rest, also in XCP-NG, it installed, but didn't generate an entry in /etc/grub2.cfg though I see its entries in /boot/loader/entries and the 4.18.0-240.15.1.el8_3 remains the default per grub2.cfg

I pestered a friend about this, who tried a test system of his (in VMware) and now his system comes up at a grub rescue prompt.

Just curious how others are faring.

cmpalmer
Posts: 3
Joined: 2021/04/09 22:56:45

Re: kernel-4.18.0-240.22.1.el8_3.x86_64 installation trouble?

Post by cmpalmer » 2021/04/09 23:08:50

I am said friend. My specific error after updating to 4.18.0-240.22.1.el8_3 was:

Code: Select all

error: ../../grub-core/kern/dl.c:431:symbol 'grub_calloc' not found.
Entering rescue mode...
grub rescue> 
Booting from the minimal 8.3 ISO, going to rescue mode, and downgrading to the previous kernel (4.18.0-240.15.1.el8_3.x86_64) allowed me to boot back into a working system. As mentioned above, this is a virtual machine guest in VMWare, specifically VMware ESXi, 6.7.0. Please let me know if providing other details can help.

donsizemore
Posts: 4
Joined: 2021/04/09 19:51:26

Re: kernel-4.18.0-240.22.1.el8_3.x86_64 installation trouble?

Post by donsizemore » 2021/04/12 12:34:01

Now that it's no longer Friday afternoon local time I'm beginning to poke at this again. dmesg doesn't tell me much, but I do see:

Code: Select all

$ sudo grubby --default-kernel
/boot/vmlinuz-4.18.0-240.22.1.el8_3.x86_64
$ uname -r
4.18.0-240.15.1.el8_3.x86_64
which smells like an issue with BLSCFG. A previous post in this forum suggested disabling BLSCFG in grub2.cfg, but that seems like a step backwards, and previous CentOS8 kernels have applied cleanly to these same systems.

User avatar
TrevorH
Site Admin
Posts: 33191
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: kernel-4.18.0-240.22.1.el8_3.x86_64 installation trouble?

Post by TrevorH » 2021/04/12 12:58:42

I've not managed to disable BLSCFG at all. I tried, it reverts it for you.

What is the output from rpm -qa grub\* ?
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

cmpalmer
Posts: 3
Joined: 2021/04/09 22:56:45

Re: kernel-4.18.0-240.22.1.el8_3.x86_64 installation trouble?

Post by cmpalmer » 2021/04/12 13:05:57

In my case, on the relevant system:

Code: Select all

# rpm -qa grub\*
grub2-tools-extra-2.02-90.el8_3.1.x86_64
grub2-tools-2.02-90.el8_3.1.x86_64
grub2-pc-modules-2.02-90.el8_3.1.noarch
grub2-tools-efi-2.02-90.el8_3.1.x86_64
grubby-8.40-41.el8.x86_64
grub2-tools-minimal-2.02-90.el8_3.1.x86_64
grub2-common-2.02-90.el8_3.1.noarch
grub2-pc-2.02-90.el8_3.1.x86_64

User avatar
TrevorH
Site Admin
Posts: 33191
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: kernel-4.18.0-240.22.1.el8_3.x86_64 installation trouble?

Post by TrevorH » 2021/04/12 13:24:44

Well here's a similar error though on Ubuntu so you'd need to check the solution is applicable:

https://askubuntu.com/questions/1263125 ... -not-found
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

donsizemore
Posts: 4
Joined: 2021/04/09 19:51:26

Re: kernel-4.18.0-240.22.1.el8_3.x86_64 installation trouble?

Post by donsizemore » 2021/04/12 13:52:37

Code: Select all

$ rpm -qa |grep grub2 |sort
grub2-common-2.02-90.el8_3.1.noarch
grub2-pc-2.02-90.el8_3.1.x86_64
grub2-pc-modules-2.02-90.el8_3.1.noarch
grub2-tools-2.02-90.el8_3.1.x86_64
grub2-tools-extra-2.02-90.el8_3.1.x86_64
grub2-tools-minimal-2.02-90.el8_3.1.x86_64
In comparing the three successful machines against the others, the problem seems to be that /etc/grub2.cfg doesn't pick up the blscfg bits from /etc/grub.d/10_linux but I don't yet understand why.

donsizemore
Posts: 4
Joined: 2021/04/09 19:51:26

Re: kernel-4.18.0-240.22.1.el8_3.x86_64 installation trouble?

Post by donsizemore » 2021/04/12 17:21:22

grub2-tools provides a

Code: Select all

grub2-switch-to-blscfg
binary that fixed my first test system, but... why didn't this happen during the 8.3 upgrade (or whenever BLSCFG became the default)? and now that I think about it, a clean 8.3 install failed to select the updated 4.18.0-240.22.1.el8_3 kernel when that was honestly the first patch applied since I stood up the VM.

User avatar
TrevorH
Site Admin
Posts: 33191
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: kernel-4.18.0-240.22.1.el8_3.x86_64 installation trouble?

Post by TrevorH » 2021/04/12 18:00:41

BLSCFG has been default since 8.0 dropped.
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

cmpalmer
Posts: 3
Joined: 2021/04/09 22:56:45

Re: kernel-4.18.0-240.22.1.el8_3.x86_64 installation trouble?

Post by cmpalmer » 2021/04/12 18:17:27

TrevorH wrote:
2021/04/12 13:24:44
Well here's a similar error though on Ubuntu so you'd need to check the solution is applicable:

https://askubuntu.com/questions/1263125 ... -not-found
From that page:
All flavours of solution involve reinstalling grub to ensure that the two parts are aligned.
I did this to pick up the new kernel packages (again):

Code: Select all

dnf -y update
... followed by this to reinstall all grub bits:

Code: Select all

dnf reinstall grub*
... after which I reviewed the file /boot/grub2/grub.cfg and confirmed I saw:

Code: Select all

### BEGIN /etc/grub.d/10_linux ###
insmod part_msdos
insmod ext2
set root='hd0,msdos1'
if [ x$feature_platform_search_hint = xy ]; then
  search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1 --hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1 --hint='hd0,msdos1'  [uuid redacted]
else
  search --no-floppy --fs-uuid --set=root [uuid redacted]
fi
insmod part_msdos
insmod ext2
set boot='hd0,msdos1'
if [ x$feature_platform_search_hint = xy ]; then
  search --no-floppy --fs-uuid --set=boot --hint-bios=hd0,msdos1 --hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1 --hint='hd0,msdos1'  [uuid redacted]
else
  search --no-floppy --fs-uuid --set=boot [uuid redacted]
fi

# This section was generated by a script. Do not modify the generated file - all changes
# will be lost the next time file is regenerated. Instead edit the BootLoaderSpec files.
#
# The blscfg command parses the BootLoaderSpec files stored in /boot/loader/entries and
# populates the boot menu. Please refer to the Boot Loader Specification documentation
# for the files format: https://www.freedesktop.org/wiki/Specifications/BootLoaderSpec/.

set default_kernelopts="root=/dev/mapper/cl-root ro crashkernel=auto resume=/dev/mapper/cl-swap rd.lvm.lv=cl/root rd.lvm.lv=cl/swap rhgb quiet "

insmod blscfg
blscfg
### END /etc/grub.d/10_linux ###
And then I rebooted with a VMware console at the ready. With some relief, I saw it come back up cleanly on the 4.18.0-240.22.1 kernel.

What is still a mystery to me is why/how this happened in the first place. I don't want to be in the habit of reinstalling grub after every kernel update to avoid this, but short of that, what sort of confidence can I have that something like this won't happen to me again on a future update of CentOS 8 or some other distro downstream of RHEL 8?

Post Reply