This procedure describes how to migrate the OS disk from internal storage to a SAN disk in a DEL R720.
While the procedure has been developped for a DEL machine, it can be applied to any other machine model, the only difference will be the RAID controller or the way to disable the internal disk.
*Note!!!: We must check with Storage the SAN disk that will be used for SAN boot is assigned with ID 0, it means the first of the SAN disks attached, whether there were some previous disks attached, they must removed all of them, assign the one for SAN boot as the first and then add the rest of disks..
Poweron the server, or init 6 whether it was already up…
Disable one internal disk to have a valid copy of the internal disk, we will work with the first disk, having second as backup.
Press CTRL+R on boot in here…
Select the second disk press F2, and choose Force offline in the disk 00:01:01.
Disks will appear like that (with 00:01:00 Online and 00:01:01 offline):
Exit and restart the server by pressing Ctrl+Alt+Del.
Startup the machine and do login.
Check current boot device.
root@server:/root# fdisk -l /dev/sda Disk /dev/sda: 146.1 GB, 146163105792 bytes 255 heads, 63 sectors/track, 17769 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sda1 * 1 13 104391 83 Linux /dev/sda2 14 17769 142625070 8e Linux LVM
Check current boot size and filesystem type:
/dev/sda1 99M 21M 73M 23% /boot /dev/sda1 on /boot type ext3 (rw)
add a new /boot and LVM partitions:
# fdisk /dev/emcpowera Partition number (1-4): 1 First cylinder (1-34526, default 1): 1 Last cylinder or +size or +sizeM or +sizeK (1-34526, default 34526): +99M --> this size has been taken from mount output previously
Add the LVM with the rest of the disk.
Partition number (1-4): 2 First cylinder (14-34526, default 14): Using default value 14 Last cylinder or +size or +sizeM or +sizeK (14-34526, default 34526): Using default value 34526
and change type to be: 8e Linux LVM
Save the current conf and check it.
# fdisk -l /dev/emcpowera Disk /dev/emcpowera: 36.2 GB, 36203397120 bytes 64 heads, 32 sectors/track, 34526 cylinders Units = cylinders of 2048 * 512 = 1048576 bytes Device Boot Start End Blocks Id System /dev/emcpowera1 1 95 97264 83 Linux /dev/emcpowera2 96 34526 35257344 8e Linux LVM #
Create FS and mount it.
# mkfs.ext3 /dev/emcpowera1 # mount /dev/emcpowera1 /mnt/temporal
Sync data: # rsync -a /boot/ /mnt/temporal # umount /mnt/temporal
Mount the new /boot
# umount /dev/sda1 # mount /dev/emcpowera1 /boot/
Check BLK and LABEL:
# blkid /dev/mapper/rootvg-swap: TYPE="swap" /dev/mapper/rootvg-home: UUID="906af97e-7ce2-4374-9a56-c9540bddcf17" TYPE="ext3" /dev/mapper/rootvg-rccm: UUID="be4d807c-7508-4962-8643-e8f01bf825a1" TYPE="ext3" /dev/mapper/rootvg-tmp: UUID="80f673e0-270c-46a1-945b-77ccd2c0c23e" TYPE="ext3" /dev/mapper/rootvg-var: UUID="672ea1a2-ef53-459b-bd37-b72e760721a9" TYPE="ext3" /dev/mapper/rootvg-root: UUID="27421769-d4e3-45fb-9b27-027274445c49" TYPE="ext3" /dev/sda1: LABEL="/boot" UUID="ff07d089-7994-4ae9-9fce-6b272d9eb8d5" TYPE="ext3" /dev/rootvg/root: UUID="27421769-d4e3-45fb-9b27-027274445c49" TYPE="ext3" /dev/rootvg/swap: TYPE="swap"
Clone UUID and label
# tune2fs /dev/emcpowera1 -U ff07d089-7994-4ae9-9fce-6b272d9eb8d5 --> from boot # tune2fs tune2fs /dev/emcpowera1 -L /boot
Add a filter in lvm.conf, check which devices are emcpower ones and add them in this file..
filter = [ "a/sda[1-9]$/", "r/sd.*/","r/disk.*/", "a/emcpower.*/"]
remove chache and reload:
# rm /etc/lvm/cache/.cache rm: remove regular file `/etc/lvm/cache/.cache'? y # pvscan; vgscan; lvscan # lvmdiskscan # pvcreate /dev/emcpowera2 Writing physical volume data to disk "/dev/emcpowera2" Physical volume "/dev/emcpowera2" successfully created
*NOTE: IMPORTANT!! Be sure the device added is emcpowera
root@server1:/root# pvs PV VG Fmt Attr PSize PFree /dev/emcpowera2 lvm2 a-- 33.59G 16.72G /dev/sda2 rootvg lvm2 a-- 136.02G 136.02G
Extend current rootvg with emcpowera2
# vgextend /dev/rootvg /dev/emcpowera2 Volume group "rootvg" successfully extended # pvmove --test --verbose /dev/sda2 /dev/emcpowera2 Test mode: Metadata will NOT be updated and volumes will not be (de)activated. Finding volume group "rootvg" Test mode: Skipping archiving of volume group. Creating logical volume pvmove0 Moving 256 extents of logical volume rootvg/root Moving 128 extents of logical volume rootvg/var Moving 32 extents of logical volume rootvg/tmp Moving 32 extents of logical volume rootvg/rccm Moving 28 extents of logical volume rootvg/home Moving 64 extents of logical volume rootvg/swap Found volume group "rootvg" activation/volume_list configuration setting not defined, checking only host tags for rootvg/root Found volume group "rootvg" activation/volume_list configuration setting not defined, checking only host tags for rootvg/var Found volume group "rootvg" activation/volume_list configuration setting not defined, checking only host tags for rootvg/tmp Found volume group "rootvg" activation/volume_list configuration setting not defined, checking only host tags for rootvg/rccm Found volume group "rootvg" activation/volume_list configuration setting not defined, checking only host tags for rootvg/home Found volume group "rootvg" activation/volume_list configuration setting not defined, checking only host tags for rootvg/swap Updating volume group metadata Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Test mode: Skipping backup of volume group. Test mode: Wiping internal cache Wiping internal VG cache
Now Move it:
# pvmove --verbose /dev/sda2 /dev/emcpowera2 Finding volume group "rootvg" Archiving volume group "rootvg" metadata (seqno 10). Creating logical volume pvmove0 Moving 256 extents of logical volume rootvg/root Moving 128 extents of logical volume rootvg/var Moving 32 extents of logical volume rootvg/tmp Moving 32 extents of logical volume rootvg/rccm Moving 28 extents of logical volume rootvg/home Moving 64 extents of logical volume rootvg/swap Found volume group "rootvg" activation/volume_list configuration setting not defined, checking only host tags for rootvg/root Found volume group "rootvg" activation/volume_list configuration setting not defined, checking only host tags for rootvg/var Found volume group "rootvg" activation/volume_list configuration setting not defined, checking only host tags for rootvg/tmp Found volume group "rootvg" activation/volume_list configuration setting not defined, checking only host tags for rootvg/rccm Found volume group "rootvg" activation/volume_list configuration setting not defined, checking only host tags for rootvg/home Found volume group "rootvg" activation/volume_list configuration setting not defined, checking only host tags for rootvg/swap Updating volume group metadata Found volume group "rootvg" Found volume group "rootvg" Creating rootvg-pvmove0 Loading rootvg-pvmove0 table (253:6) Loading rootvg-root table (253:0) Loading rootvg-var table (253:1) Loading rootvg-tmp table (253:2) Loading rootvg-rccm table (253:3) Loading rootvg-home table (253:4) Loading rootvg-swap table (253:5) Suspending rootvg-root (253:0) with device flush Suspending rootvg-pvmove0 (253:6) with device flush Suspending rootvg-var (253:1) with device flush Suspending rootvg-tmp (253:2) with device flush Suspending rootvg-rccm (253:3) with device flush Suspending rootvg-home (253:4) with device flush Suspending rootvg-swap (253:5) with device flush Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" activation/volume_list configuration setting not defined, checking only host tags for rootvg/pvmove0 Resuming rootvg-pvmove0 (253:6) Found volume group "rootvg" Loading rootvg-pvmove0 table (253:6) Suppressed rootvg-pvmove0 identical table reload. Resuming rootvg-root (253:0) Resuming rootvg-var (253:1) Resuming rootvg-tmp (253:2) Resuming rootvg-rccm (253:3) Resuming rootvg-home (253:4) Resuming rootvg-swap (253:5) Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Creating volume group backup "/etc/lvm/backup/rootvg" (seqno 11). Checking progress before waiting every 15 seconds /dev/sda2: Moved: 0.0% /dev/sda2: Moved: 39.8% Updating volume group metadata Found volume group "rootvg" Found volume group "rootvg" Loading rootvg-pvmove0 table (253:6) Suspending rootvg-pvmove0 (253:6) with device flush Found volume group "rootvg" Resuming rootvg-pvmove0 (253:6) Creating volume group backup "/etc/lvm/backup/rootvg" (seqno 12). /dev/sda2: Moved: 61.1% /dev/sda2: Moved: 71.1% Updating volume group metadata Found volume group "rootvg" Found volume group "rootvg" Loading rootvg-pvmove0 table (253:6) Suspending rootvg-pvmove0 (253:6) with device flush Found volume group "rootvg" Resuming rootvg-pvmove0 (253:6) Creating volume group backup "/etc/lvm/backup/rootvg" (seqno 13). /dev/sda2: Moved: 77.0% Updating volume group metadata Found volume group "rootvg" Found volume group "rootvg" Loading rootvg-pvmove0 table (253:6) Suspending rootvg-pvmove0 (253:6) with device flush Found volume group "rootvg" Resuming rootvg-pvmove0 (253:6) Creating volume group backup "/etc/lvm/backup/rootvg" (seqno 14). /dev/sda2: Moved: 83.0% Updating volume group metadata Found volume group "rootvg" Found volume group "rootvg" Loading rootvg-pvmove0 table (253:6) Suspending rootvg-pvmove0 (253:6) with device flush Found volume group "rootvg" Resuming rootvg-pvmove0 (253:6) Creating volume group backup "/etc/lvm/backup/rootvg" (seqno 15). /dev/sda2: Moved: 88.1% Updating volume group metadata Found volume group "rootvg" Found volume group "rootvg" Loading rootvg-pvmove0 table (253:6) Suspending rootvg-pvmove0 (253:6) with device flush Found volume group "rootvg" Resuming rootvg-pvmove0 (253:6) Creating volume group backup "/etc/lvm/backup/rootvg" (seqno 16). /dev/sda2: Moved: 100.0% Found volume group "rootvg" Found volume group "rootvg" Loading rootvg-root table (253:0) Loading rootvg-var table (253:1) Loading rootvg-tmp table (253:2) Loading rootvg-rccm table (253:3) Loading rootvg-home table (253:4) Loading rootvg-swap table (253:5) Loading rootvg-pvmove0 table (253:6) Suspending rootvg-root (253:0) with device flush Suspending rootvg-var (253:1) with device flush Suspending rootvg-tmp (253:2) with device flush Suspending rootvg-rccm (253:3) with device flush Suspending rootvg-home (253:4) with device flush Suspending rootvg-swap (253:5) with device flush Suspending rootvg-pvmove0 (253:6) with device flush Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Found volume group "rootvg" Resuming rootvg-pvmove0 (253:6) Found volume group "rootvg" Resuming rootvg-root (253:0) Found volume group "rootvg" Resuming rootvg-var (253:1) Found volume group "rootvg" Resuming rootvg-tmp (253:2) Found volume group "rootvg" Resuming rootvg-rccm (253:3) Found volume group "rootvg" Resuming rootvg-home (253:4) Found volume group "rootvg" Resuming rootvg-swap (253:5) Found volume group "rootvg" Removing rootvg-pvmove0 (253:6) Removing temporary pvmove LV Writing out final volume group after pvmove Creating volume group backup "/etc/lvm/backup/rootvg" (seqno 18). root@server1:/root#
Now remove the internal disk from the VLM volume
# vgreduce /dev/rootvg /dev/sda2 Removed "/dev/sda2" from volume group "rootvg"
Install GRUB:
# grub Probing devices to guess BIOS drives. This may take a long time. GNU GRUB version 0.97 (640K lower / 3072K upper memory) [ Minimal BASH-like line editing is supported. For the first word, TAB lists possible command completions. Anywhere else TAB lists the possible completions of a device/filename.] grub> device (hd0) /dev/emcpowera device (hd0) /dev/emcpowera grub> root (hd0,0) root (hd0,0) Filesystem type is ext2fs, partition type 0x83 grub> setup (hd0) setup (hd0) Checking if "/boot/grub/stage1" exists... no Checking if "/grub/stage1" exists... yes Checking if "/grub/stage2" exists... yes Checking if "/grub/e2fs_stage1_5" exists... yes Running "embed /grub/e2fs_stage1_5 (hd0)"... 15 sectors are embedded. succeeded Running "install /grub/stage1 (hd0) (hd0)1+15 p (hd0,0)/grub/stage2 /grub/grub.conf"... succeeded Done. grub>
Create a new initrd image (Note, this is just in case, the normal initrd doesn’t work due to EMCpower devices, probably it won’t be needed).
/sbin/mkinitrd -v -f /boot/initrd-`uname -r`-EMC.img `uname -r`
OR
Check in menu.lst current loaded initrd.
title Red Hat Enterprise Linux Server (2.6.18-308.4.1.el5) root (hd0,0) kernel /vmlinuz-2.6.18-308.4.1.el5 ro root=/dev/rootvg/root biosdevname=0 initrd /initrd-2.6.18-308.4.1.el5.img
Create a new initrd (just in case)
# /sbin/mkinitrd -v -f /boot/initrd-2.6.18-308.4.1.el5-emc.img 2.6.18-308.4.1.el5
Restart the server and go to BIOS… (F2) to disable the internal RAID Card.
Exit saving…
Go to SAN Card configuration.
SAN Card CTRL+Q
esc, esc, save!!
there are two SAN cards.. we must enable them in boot both of them. Repeat it in the second SAN CARD, and reboot the system.
after start the machine, it must show this in the boot up screen…
Startup normally the system.
After boot:
root@server1:/root# df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/rootvg-root 7.8G 1.7G 5.8G 23% / /dev/mapper/rootvg-var 3.9G 234M 3.5G 7% /var /dev/mapper/rootvg-tmp 992M 34M 908M 4% /tmp /dev/mapper/rootvg-rccm /dev/sda1 92M 25M 63M 29% /boot root@server1:/root# pvs Found duplicate PV Y89Fwy98EU9oSBeBxvlTmFn2Jz9k3tW2: using /dev/emcpowera2 not /dev/sda2 PV VG Fmt Attr PSize PFree /dev/emcpowera2 rootvg lvm2 a-- 33.59G 16.72G root@server1:/root# fdisk -l Disk /dev/sda: 36.2 GB, 36203397120 bytes 64 heads, 32 sectors/track, 34526 cylinders Units = cylinders of 2048 * 512 = 1048576 bytes Device Boot Start End Blocks Id System /dev/sda1 1 95 97264 83 Linux /dev/sda2 96 34526 35257344 8e Linux LVM Disk /dev/sdb: 36.2 GB, 36203397120 bytes 64 heads, 32 sectors/track, 34526 cylinders Units = cylinders of 2048 * 512 = 1048576 bytes Device Boot Start End Blocks Id System /dev/sdb1 1 95 97264 83 Linux /dev/sdb2 96 34526 35257344 8e Linux LVM Disk /dev/emcpowera: 36.2 GB, 36203397120 bytes 64 heads, 32 sectors/track, 34526 cylinders Units = cylinders of 2048 * 512 = 1048576 bytes Device Boot Start End Blocks Id System /dev/emcpowera1 1 95 97264 83 Linux /dev/emcpowera2 96 34526 35257344 8e Linux LVM
If you need to undone and return to boot from local disk.
Common steps…
Enter BIOS with F2 on boot, enter System BIOS, Integrated Devices and enable RAID controller again.
Exit saving, now, the controller and the disk will be available again on boot to select Ctrl+R
Enter SAN HBA card Configuration (Ctrl + Q), and disable the features enable in the beginning.
Configuration settings –> Adapter settings –> Host adapter BIOS –> Disabled
Configuration settings –> Selectable Boot settings –> Selectable Boot –> Disabled
–> Boot Port Name (press C)
Exit saving, and repeat it in the second HBA.
Exit saving, and Reboot System.
Enter the RAID controller (Power Edged Expandable RAID configuration) (Ctrl+R), disable the first disk called 00:01:00 (F2 and then Force Offline),
enable the second disk 00:01:01 (F2 and Force Online).
The disks will be showed like…
Now, select the disk 00:01:00 and press f2, and choose Rebuild.
(***Note: You have two options now. If you don’t trust yourself enough, wait until the rebuild is finished and perform the mirror break in the beginning of this document, you will have again two valid copied of data in case something goes wrong, or, in case you know what you’re doing, continue exiting from HBA card configuration but please, note, there will be only one disk with valid data).
Exit and restart the machine with Ctrl + Alt + Del.
Check the boot of the server, and notice whether the following outputs appear.
ROM BIOS NOT INSTALLED in regards of the HBA card…
And PowerEdge RAID controller is again available with 1 Virtual disk in degraded state…
### End common steps… ##########
Two options:
In case the SAN disk data is valid, or the Local disk data is not updated then….
(*Note: you are going to migrate data to local disk, we must assure the SAN disk data is valid!!!)
Poweron or restart the physical server
Boot the server normally, and add the sda disk to rootvg.
# vgextend /dev/rootvg /dev/sda2 # pvmove --verbose /dev/emcpowera2 /dev/sda2 # vgreduce /dev/rootvg /dev/emcpowera2 # pvremove /dev/emcpowera2 Labels on physical volume "/dev/emcpowera2" successfully wiped # grub Probing devices to guess BIOS drives. This may take a long time. GNU GRUB version 0.97 (640K lower / 3072K upper memory) [ Minimal BASH-like line editing is supported. For the first word, TAB lists possible command completions. Anywhere else TAB lists the possible completions of a device/filename.] grub> device (hd0) /dev/sda device (hd0) /dev/sda grub> root (hd0,0) root (hd0,0) Filesystem type is ext2fs, partition type 0x83 grub> setup (hd0) setup (hd0) Checking if "/boot/grub/stage1" exists... no Checking if "/grub/stage1" exists... yes Checking if "/grub/stage2" exists... yes Checking if "/grub/e2fs_stage1_5" exists... yes Running "embed /grub/e2fs_stage1_5 (hd0)"... 15 sectors are embedded. succeeded Running "install /grub/stage1 (hd0) (hd0)1+15 p (hd0,0)/grub/stage2 /grub/grub.conf"... succeeded Done. grub>
Second option, in case the SAN disk data is not usable and we want to recover from the available copy of local disks.
(Note**: this data won’t most probably won’t be update, bear it on mind).
In Virtual Media, add the RH iso image to boot the server (like RHEL5_U8_64bit.iso).
When will appear this screen
choose Bios Boot Manager by pressing F11.
When the screen will STOP, choose, BIOS Boot Manager option, and Virtual CD.
and write “linux rescue” (F5) and, at some point, when you will be requested, skip start network interfaces and looking up for root filesystem.
Now, run
# fdisk -l Note there is one internal disk sda with at least 146Gb and some other (depending on the SAN attached disks). The important information is sda must be with those two partitions Linux and Linux LVM. /dev/sda1 1 95 XXXXX 83 Linux /dev/sda2 96 34526 YYYYYY 8e Linux LVM # /sbin/rmmod qla2xxx # fdisk -l
you should see only sda
Now run:
# lvm pvs --> note rootvg belongs to sda2 # lvm vgchange -a y
and mount rootvg/root in a temporary folder:
# mkdir /mnt/temporary # mount /dev/rootvg/root /mnt/temporary
Check whether the data is valid…
and now
# /sbin/modprobe qla2xxx.ko # fdisk /dev/sdb and remove all partitions ( by pressing o).
Now restart the server, write exit, and boot normally.
Now it removed the sda disk from rootvg.
# lvm pvs /dev/sda2 # lvm vgcfgbackup -f /tmp/vgconf rootvg
Check in /tmp/vgconf what is the pv0 id and run the following command.
# lvm pvcreate -u <missing PV UUID> --restorefile /tmp/vgconf rootvg # lvm vgcfgrestore -f /tmp/vgcfg rootvg # lvm vgchange -ay # /sbin/modprobe qla2xxx.ko
mount rootvg2/root in a temporary folder
go to /etc/lvm/archive and choose one file that have the correct sda configuration. The file that contains this line is the correct one.
Copy it to /tmp, umount the temporary folder, and run
# lvm vgchange -a n
now, use the backup file with the rootvg configuration:
As an example:
# lvm pvs --> write down the disk included in rootvg # lvm vgextend /dev/rootvg /dev/sda2 # pvmove --verbose /dev/sdc2 /dev/sda2
Remove all the partitions in /dev/emcpower (formerly /dev/sdb, /dev/sdc), remount / with rw, and mount /dev/sda1 in /boot
From “Virtual Media” add the new image from RH… boot the server, choose CDRom and select rescue image.
