Boot stop after: Reached target Path Units

Issues related to applications and software problems and general support
Post Reply
lee22
Posts: 4
Joined: 2020/10/10 07:15:42

Boot stop after: Reached target Path Units

Post by lee22 » 2023/12/02 13:01:41

Every new kernel after 5.14.0-375 did not boot.
After "Reached target Path Units" boot process is freezed. Only cure I got is go back to 5.14.0-375 kernel.
I have 3 different notebooks (ASUS) with CentOS 9 - only one have this problem.
Any ideas what going on?

User avatar
TrevorH
Site Admin
Posts: 33263
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: Boot stop after: Reached target Path Units

Post by TrevorH » 2023/12/03 11:34:59

Edit the kernel comand line at the grub menu and remove 'rhgb' and 'quiet' and then continue with the boot. It will not change the outcome but you should now be able to see any error messages issued that were previously invisible.
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

NorthStar
Posts: 3
Joined: 2023/12/05 16:55:51

Re: Boot stop after: Reached target Path Units

Post by NorthStar » 2023/12/05 17:22:08

I have the same problem. My guess is that the notebook you’re having problems with is more recent than the other two.

The machine I’m having this problem with is a HP Z2 Tower G9 Workstation Desktop PC with an Alder Lake-S 12th Gen Intel Core i9-12900K. The main disk drive is a 6TB Seagate ST6000NM019B-2TG103 attached to the built-in SATA controller. CentOS is installed on this HDD. This is the device entry for this controller:

Code: Select all

10000:e0:17.0 SATA controller [0106]: Intel Corporation Alder Lake-S PCH SATA Controller [AHCI Mode] [8086:7ae2] (rev 11)
Boot fails because the CentOS kernel cannot “see” the controller, therefore the dev links cannot be built. If you wait long enough (a few minutes) you’ll eventually see a number of messages to that effect. The last kernel I can boot with is 5.14.0-375.el9.x86_64.

Here’s an interesting take on the installed kernels:

Code: Select all

222M	/lib/modules/5.14.0-375.el9.x86_64
130M	/lib/modules/5.14.0-390.el9.x86_64
Notice how much smaller 5.14.0-390.el9.x86_64, the latest kernel, is? I think it may be due to a regression in the CentOS kernel where a number of drivers seem to have been omitted, and this for every kernel released since 375. This problem won’t be resolved until the developers notice that they've somehow misplaced the drivers for recent Intel processors.

User avatar
TrevorH
Site Admin
Posts: 33263
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: Boot stop after: Reached target Path Units

Post by TrevorH » 2023/12/05 18:14:12

You should report this on issues.redhat.com under RHEL then I think there is a version number "Stream 9" or something like that. No-one from RH reads this forum so no-one who either cares or can fix it will ever see these posts.

There is only one driver for all SATA chipsets and that is the ahci module. If you use lsinitrd on the initramfs file for the 390 kernel and grep for ahci, does it have it included? If it does not then you may be able to force dracut to include it and rebuild it. I suspect it's there and something else is going on - I rather suspect the pci id that you posted is "strange" and maybe it's not checking the device at all: 10000:e0:17.0 looks very odd. I jsut checked the most recent hardware that I have with the longest list of PCI ids and not one of them is in that format.
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

NorthStar
Posts: 3
Joined: 2023/12/05 16:55:51

Re: Boot stop after: Reached target Path Units

Post by NorthStar » 2023/12/05 20:28:03

Understood. I will try posting this to issues.redhat.com.

The lsinitrd yields surprising results:

Code: Select all

sudo lsinitrd /boot/initramfs-5.14.0-390.el9.x86_64.img | fgrep ahci
-rw-r--r--   1 root     root        18136 Nov 22 12:18 usr/lib/modules/5.14.0-390.el9.x86_64/kernel/drivers/ata/ahci.ko.xz
-rw-r--r--   1 root     root        26372 Nov 22 12:18 usr/lib/modules/5.14.0-390.el9.x86_64/kernel/drivers/ata/libahci.ko.xz

sudo lsinitrd /boot/initramfs-5.14.0-375.el9.x86_64.img | fgrep ahci
-rw-r--r--   1 root     root        18088 Oct  9 12:43 usr/lib/modules/5.14.0-375.el9.x86_64/kernel/drivers/ata/ahci.ko.xz
-rw-r--r--   1 root     root        26320 Oct  9 12:43 usr/lib/modules/5.14.0-375.el9.x86_64/kernel/drivers/ata/libahci.ko.xz
Both the booting 375 kernel and the non-booting 390 kernel show the same results. However, every kernel since 375 (378, 381, 383, 386, 388 and the latest 390) have not managed to “find” the controller and build the dev entries for it. If I wait long enough, I get failed “-f” tests for the device files and timeout messages from dracut-initqueue, and eventually fall into a dracut emergency shell.

As the for PCI ids, I agree with you; I’ve never seen such entries before. I have an older HP Workstation and its ids are all of the “0000:” format, but those “10000:” are what the Z2G9 Workstation first booted with. I haven’t had any problems with most Linux distributions, and I run four others along with CentOS Stream. All they require is that the kernel be more recent than 5.18/5.19, since that’s when Intel added the patches for 12th Gen Alder Lake-S processors, including the thread director to properly balance tasks between the so-called “performance” and “efficiency” cores. This used to work obviously, since I couldn’t have installed CentOS in the first place.

Finally, I am not very proficient with dracut; all I use if for is to occasionally regenerate the initramfs files when needed. Would you know a good resource on how to include and rebuild drivers?

User avatar
TrevorH
Site Admin
Posts: 33263
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: Boot stop after: Reached target Path Units

Post by TrevorH » 2023/12/06 00:41:39

I just booted my Dell XPS 13 Plus 9320 - 12th gen i5-1240P (4 and 8 cores) and my lspci there is not showing those strange entries, just the normal ones of the format 01:00.0 or 00:1f.5. I have never seen PCI entries like yours before - they appear to have an extra 5 digit number prefixed at the start. Got no idea if that has anything at all to do with the problem.

Did you remove rhgb quiet from the kernel command line as I suggested to an earlier poster on this thread? Will not fix the problem but might give you some more idea about what is going wrong. You might want to have a fast frame video camera to capture what flies past as it will be done in a flash. It may be possible to run `dmesg -T` from the emergency shell and pipe that to a file for later - possibly to a USB stick if your filesystems are all r/o at that stage. You may be able to tell more by comparing that with the same output from the last working kernel.

You don't need to use dracut to rebuild anything as both your current in initramfs files contain the ahci module. I am surprised the files are as large as they are, just checked and mine (Fedora 388) are all 38MB and even the rescue kernel one is "only" 72MB. Positively slim compared to your 222 or 130MB ones!

You could double check your BIOS is set up to use AHCI and not RAID for the SATA ports perhaps? I think that's unlikely but...
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

NorthStar
Posts: 3
Joined: 2023/12/05 16:55:51

Re: Boot stop after: Reached target Path Units

Post by NorthStar » 2023/12/06 19:49:27

This is some of the inxi data on my HP Z2 Tower G9 Workstation PC:

Code: Select all

System:
  Kernel: 5.14.0-375.el9.x86_64 arch: x86_64 bits: 64 compiler: gcc
    v: 2.35.2-42.el9 Desktop: GNOME v: 40.10 Distro: CentOS Stream release 9
    base: RHEL 9
Machine:
  Type: Desktop System: HP product: HP Z2 Tower G9 Workstation Desktop PC
    v: N/A serial: <superuser required>
  Mobo: HP model: 895C v: KBC Version 12.03.05 serial: <superuser required>
    UEFI: HP v: U50 Ver. 02.03.02 date: 08/31/2023
CPU:
  Info: 16-core (8-mt/8-st) model: 12th Gen Intel Core i9-12900K bits: 64
    type: MST AMCP arch: Alder Lake rev: 2 cache: L1: 1.4 MiB L2: 14 MiB
    L3: 30 MiB
  Speed (MHz): avg: 794 high: 801 min/max: 800/5100:5200:3900 cores: 1: 801
    2: 801 3: 690 4: 801 5: 800 6: 801 7: 800 8: 801 9: 800 10: 800 11: 800
    12: 801 13: 800 14: 801 15: 800 16: 759 17: 800 18: 800 19: 800 20: 801
    21: 800 22: 799 23: 800 24: 800 bogomips: 152985
  Flags: avx avx2 ht lm nx pae sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx
Graphics:
  Device-1: NVIDIA TU117GL [T1000 8GB] driver: nvidia v: 545.29.06
    arch: Turing bus-ID: 0000:01:00.0
  Display: x11 server: X.Org v: 1.20.11 with: Xwayland v: 22.1.9 driver: X:
    loaded: nvidia gpu: nvidia resolution: 1: 1920x1200~60Hz 2: 1920x1200~60Hz
  API: OpenGL v: 4.6.0 vendor: nvidia v: 545.29.06 glx-v: 1.4
    direct-render: yes renderer: NVIDIA T1000 8GB/PCIe/SSE2
  API: EGL Message: EGL data requires eglinfo.
And these are the infamous lspci entries:

Code: Select all

0000:00:00.0 Host bridge [0600]: Intel Corporation 12th Gen Core Processor Host Bridge/DRAM Registers [8086:4660] (rev 02)
0000:00:01.0 PCI bridge [0604]: Intel Corporation 12th Gen Core Processor PCI Express x16 Controller #1 [8086:460d] (rev 02)
0000:00:06.0 System peripheral [0880]: Intel Corporation RST VMD Managed Controller [8086:09ab]
0000:00:0e.0 RAID bus controller [0104]: Intel Corporation Volume Management Device NVMe RAID Controller [8086:467f]
0000:00:14.0 USB controller [0c03]: Intel Corporation Alder Lake-S PCH USB 3.2 Gen 2x2 XHCI Controller [8086:7ae0] (rev 11)
0000:00:14.2 RAM memory [0500]: Intel Corporation Alder Lake-S PCH Shared SRAM [8086:7aa7] (rev 11)
0000:00:16.0 Communication controller [0780]: Intel Corporation Alder Lake-S PCH HECI Controller #1 [8086:7ae8] (rev 11)
0000:00:17.0 System peripheral [0880]: Intel Corporation RST VMD Managed Controller [8086:09ab]
0000:00:1c.0 PCI bridge [0604]: Intel Corporation Alder Lake-S PCH PCI Express Root Port #5 [8086:7abc] (rev 11)
0000:00:1c.7 PCI bridge [0604]: Intel Corporation Alder Lake-S PCH PCI Express Root Port #8 [8086:7abf] (rev 11)
0000:00:1f.0 ISA bridge [0601]: Intel Corporation Device [8086:7a88] (rev 11)
0000:00:1f.3 Audio device [0403]: Intel Corporation Alder Lake-S HD Audio Controller [8086:7ad0] (rev 11)
0000:00:1f.4 SMBus [0c05]: Intel Corporation Alder Lake-S PCH SMBus Controller [8086:7aa3] (rev 11)
0000:00:1f.5 Serial bus controller [0c80]: Intel Corporation Alder Lake-S PCH SPI Controller [8086:7aa4] (rev 11)
0000:00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (17) I219-LM [8086:1a1c] (rev 11)
0000:01:00.0 VGA compatible controller [0300]: NVIDIA Corporation TU117GL [T1000 8GB] [10de:1ff0] (rev a1)
0000:01:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:10fa] (rev a1)
0000:02:00.0 Network controller [0280]: Intel Corporation Wi-Fi 6 AX210/AX211/AX411 160MHz [8086:2725] (rev 1a)
0000:03:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:0612] (rev 02)
10000:e0:06.0 PCI bridge [0604]: Intel Corporation 12th Gen Core Processor PCI Express x4 Controller #0 [8086:464d] (rev 02)
10000:e0:17.0 SATA controller [0106]: Intel Corporation Alder Lake-S PCH SATA Controller [AHCI Mode] [8086:7ae2] (rev 11)
10000:e1:00.0 Non-Volatile memory controller [0108]: Micron Technology Inc 3400 NVMe SSD [Hendrix] [1344:5407]
I admit those last three entries prefixed with “10000:” are a bit odd, but that’s how the machine reports them. I also need to reiterate that I am not trying to make CentOS Stream 9 work; it already does, and perfectly, but only up to kernel 375. This is therefore a regression, where something that used to work no longer does. I’ll also point out again that the kernels differ significantly in size:

Code: Select all

222M	/lib/modules/5.14.0-375.el9.x86_64
130M	/lib/modules/5.14.0-390.el9.x86_64
A lot has suspiciously disappeared between 375 and 390, and the former is the last kernel that boots on this PC. I also have no idea why the initramfs files are so large; I have not altered the dracut configuration in any way.

I don’t have rhgb quiet in the kernel command line as I like to follow the booting process. As the previous poster has reported, boot stops at the “Reached target Path Units” and hangs there for a few minutes with no disk activity whatsoever. Then messages stating that “-f” tests from dracut-initqueue have failed for device files starting with “/lib/dracut/hooks/…” with timeout warnings, until finally the process stops with a dracut emergency shell. An ls command at this point reveals that there are no dev disk entries for the SATA HDD. It is clear from this that the recent post-375 kernels no longer find the SATA controller. I could collect the messages but none of them provide any clues from a hardware perspective. Kernel 375 outputs all the messages typical of a normal boot, all the others provide no boot messages as they do not boot at all.

As for changing the machine’s BIOS configuration, I have to mention that apart from CentOS Stream 9, I have those other Linux distributions running perfectly on it:
  • Fedora 39 (6.6.4_200.fc39)
  • Linux Mint 21.2 (6.2.0_37)
  • Ubuntu 23.10 (6.5.0_13)
  • Debian 13 Testing (6.5.10_1)
  • Arch Linux (6.6.4_arch1_1)
  • Mageia 9 (6.5.11_5)
  • openSUSE Tumbleweed (6.6.3_1.1)
  • Solus 4.4 (6.6.4_265)
  • CentOS Stream 9 (5.14.0_375.el9)
These are all installed in multiple partitions on the same 6TB HDD where CentOS resides. I use the machine’s UEFI boot menu to directly run the GRUB efi entry for each distro. This clearly demonstrates that the problem is with CentOS Stream 9, and only with the last 6 kernels.

As a retired IT professional with a UNIX background (Solaris, AIX, and some HP-UX) I pass the time by “collecting” Linux distributions for comparative study, and I also provide part-time support for beginners. I have recently registered as a Red Hat developer, and I have downloaded Red Hat Enterprise Linux 9.3; I plan to install it soon.

My point is that my current problem with the latest CentOS Stream 9 kernels is not with the PC but clearly lies with a recent change to how the kernel is built. I have registered an account for issues.redhat.com, but that Jira interface is one of the most unintuitive I have ever encountered; I’ll get back to it as soon as I can.

User avatar
TrevorH
Site Admin
Posts: 33263
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: Boot stop after: Reached target Path Units

Post by TrevorH » 2023/12/06 20:49:02

I admit those last three entries prefixed with “10000:” are a bit odd, but that’s how the machine reports them
The whole lot are odd. I've not got a physical machine in about 40 or 50 of them that has that extra field at the front of the PCI id. All mine are of the format

Code: Select all

07:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51)
Yours has an extra leading field that is not present on any of mine.
A lot has suspiciously disappeared between 375 and 390
I'd suggest running ls -laR /lib/modules/5.14.0-375.el9.x86_64 > /tmp/375.txt and repeat for the other modules directory, piping to a different file then diff them to see what is missing. You may need to hack the data format about a bit to remove dates/times/filesizes to get a decent diff.
boot stops at the “Reached target Path Units”
I'm pretty sure that up until then it's running out of the initramfs and getting itself set up to move to the real root directory.. If the device/controller that holds your root filesystem is not found then it's not going to get past that.
As for changing the machine’s BIOS configuration
I'm suggesting that you _check_ it to make sure. I'm pretty sure it's set to AHCI mode due to the output of lspci saying "AHCI Mode" but I think it would be wise to check.
all the others provide no boot messages as they do not boot at all.
You said they drop you to an emergency shell. You should be able to run dmesg -T from there and pipe the output to a file. Then compare that with a successful boot.

I'm not doubting the problem is with the new kernel. I am suggesting ways in which you might be able to find out more about what is going wrong.

I don't use CentOS Stream at all. It's not suitable for production use and I will not use it. I'd recommend sticking with RHEL or one of the other rebuilds if you want to avoid problems like this - CentOS Stream is a perpetual beta where RH devs use you lot as guinea pigs. I do not wish to join in.

RHEL 9.3 will boot just fine as the current RHEL 9 kernels are all kernel-5.14.0-362.* ones
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

Post Reply