Resolution
To resolve this issue, one or both of the following may be required, followed by rebuilding the initial ramdisk:
- Repair the LVM filter in /etc/lvm/lvm.conf to ensure it accepts the device associated with the root filesystem.
- Ensure that the root VG and LV paths references in the GRUB configuration are correct.
The following steps offer one approach to troubleshooting and resolving this, making use of the dracut shell:
Enter the dracut shell
The dracut shell will replace a kernel panic after an error in the initial ramdisk, allowing you to troubleshoot and possibly resolve the issue from there. To enter the dracut shell in place of the panic, interrupt the GRUB boot process, edit the kernel command line as follows, and then continue the boot process:
– Remove these (if present): `rhgb quiet`
– Add this: `rdshell`
Continue to boot with this change. After the dracut warning, you should see something like this:
Dropping to debug shell.
sh: can't access tty; job control turned off dracut:/#
View and repair the LVM filter in /etc/lvm/lvm.conf
1. From the dracut shell described in the first section, run the following commands at the prompt:
dracut:/# lvm pvs dracut:/# lvm lvs
If the root VG and LVs are shown in the output, skip to the next section on repairing the GRUB configuration. If they are missing, go on to the next step.
2. Remove the LVM configuration file in the currently running initial ramdisk. Note that this does not persist, and it does not affect the root filesystem. It is only a change in what is currently loaded in RAM for the ramdisk. This will force LVM to use its default “accept all” filter when scanning for devices:
dracut:/# rm /etc/lvm/lvm.conf
3.Re-scan devices with this LVM filter removed and confirm that the root VG was found:
dracut:/# lvm pvscan dracut:/# lvm vgscan dracut:/# lvm lvscan
4. If the root VG was found, skip this step. If the root VG was not found, check the output of blkid to confirm whether the physical block device was found. If the devices itself is not showing, examine physical connections to the device. If the device is showing, investigate whether there was corruption on the device causing LVM data to be unreadable. Reach our to Red Hat Support for assistance as needed:
dracut:/# blkid /dev/vda1: UUID="6469e70f-938b-4046-aefe-7a0c1ad1809b" TYPE="ext4" /dev/vda2: UUID="L4P8yb-1dCG-H4Je-uvpj-pNuK-VBRr-eTkmsG" TYPE="LVM2_member"
5. If the VG was found in the scans when the lvm.conf was deleted, boot to the rescue environment or to an alternate kernel (if possible) and adjust the “filter =” line in /etc/lvm/lvm.conf as appropriate. For fastest recovery, switch to the default filter so the system can boot normally. You can revisit the filter later to customize it as needed. The default “accept all” filter is shown here:
filter = [ "a/.*/" ]
6.Rebuild the initial ramdisk following changes to /etc/lvm/lvm.conf, prior to rebooting the system:How to rebuild the initial ramdisk image in Red Hat Enterprise Linux
# dracut -f -v
7. Try rebooting the system normally. If the same error occurs again, even with a corrected filter, see the next section covering the GRUB configuration.
View and repair the GRUB configuration
1. From the dracut shell described in the first section, run the “lvm pvs” and “lvm lvs” commands at the prompt. Make note of the output, specifically the device, VG, and LV names. The following shows example output with the commands:
dracut:/# lvm pvs File descriptor 9 (/.console_lock) leaked on lvm invocation. Parent PID 1683: sh PV VG Fmt Attr PSize PFree /dev/vda2 vg_stark lvm2 a-- 9.51g 0 dracut:/# lvm lvs File descriptor 9 (/.console_lock) leaked on lvm invocation. Parent PID 1674: sh LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert lv_root vg_stark -wi------- 8.51g lv_swap vg_stark -wi------- 1.00g
2. Reboot the system and interrupt the GRUB boot loader again. From there, correct the kernel line as needed to ensure that the LV and VG names match with the output shown. Then, continue booting normally.
NOTE: If the system still fails to boot normally with these changes in place, proceed with booting into the rescue environment for further troubleshooting.
3. After booting, edit the GRUB configuration to make the kernel command line changes permanent.
4. Reboot again to confirm that the GRUB changes were successful.
Root Cause
One reason for this message is that the LVM filter, set using the “filter” parameter in /etc/lvm/lvm.conf, is too restrictive. As a result, when the initial ramdisk scans for devices, it rejects the device that contains the root filesystem. Then, when the ramdisk goes on to look for LVM metatdata and the root filesystem, it is not found in the remaining devices.
Another reason for this message is that the LVM configuration on the disk is inconsistent with the configuration in the initial ramdisk or in GRUB. An example of this would be renaming the root VG or LV without updating GRUB or rebuilding the ramdisk.