Page 1 of 2

How to recover data from RAID1

Posted: 2021/09/14 19:14:04
by vicn1222
Hi,

I installed CentOS 8 in my old Dell T610 hardware RAID1 server, with driver download from https://elrepo.org/linux/dud/el8/x86_64/.

It all worked well, except today my machine lost power and can't start any more.

I try to start rescue with no success (can't find the hard drive on rescue). I pull off one of the RAID 1 disk, and plug into my other CentOS 8 machine, but it won't mount (/dev/sdc), as shown the picture. /dev/sdc1 and /dev/sdc2 can mount, but /dev/sdc3 won't mount and it is where the data have been stored.

I did google search the whole day and followed many different threads, but never worked. Please help. Thank you!

lsblk shows below:

Code: Select all

[root@t320 archive]# lsblk
NAME             MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
loop0              7:0    0  61.8M  1 loop  /var/lib/snapd/snap/core20/1081
loop1              7:1    0  42.5M  1 loop  /var/lib/snapd/snap/certbot/1343
loop2              7:2    0  99.4M  1 loop  /var/lib/snapd/snap/core/11420
loop3              7:3    0  42.3M  1 loop  /var/lib/snapd/snap/certbot/1434
loop4              7:4    0  99.4M  1 loop  /var/lib/snapd/snap/core/11606
loop5              7:5    0  61.7M  1 loop  /var/lib/snapd/snap/core20/1026
sda                8:0    0 931.5G  0 disk  
├─sda1             8:1    0 865.5G  0 part  
│ └─md125          9:125  0 865.3G  0 raid1 /
├─sda2             8:2    0  64.1G  0 part  
│ └─md126          9:126  0    64G  0 raid1 [SWAP]
├─sda3             8:3    0     1G  0 part  
│ └─md127          9:127  0  1023M  0 raid1 /boot
└─sda4             8:4    0     1G  0 part  
  └─md124          9:124  0     1G  0 raid1 /boot/efi
sdb                8:16   0 931.5G  0 disk  
├─sdb1             8:17   0 865.5G  0 part  
│ └─md125          9:125  0 865.3G  0 raid1 /
├─sdb2             8:18   0  64.1G  0 part  
│ └─md126          9:126  0    64G  0 raid1 [SWAP]
├─sdb3             8:19   0     1G  0 part  
│ └─md127          9:127  0  1023M  0 raid1 /boot
└─sdb4             8:20   0     1G  0 part  
  └─md124          9:124  0     1G  0 raid1 /boot/efi
sdc                8:32   0   1.8T  0 disk  
├─sdc1             8:33   0     1G  0 part  /run/media/lideng/09C6-A91C
├─sdc2             8:34   0     1G  0 part  /run/media/lideng/a1c606a5-6563-48f6-a91d-0cea7ca9884d
└─sdc3             8:35   0   1.8T  0 part  
  ├─cs_t610-swap 253:0    0    32G  0 lvm   
  └─cs_t610-root 253:1    0   1.8T  0 lvm   
sr0               11:0    1  1024M  0 rom   
[root@t320 archive]# 

Re: How to recover data from RAID1

Posted: 2021/09/14 19:40:15
by TrevorH
Your /dev/sdc3 is an LVM physical volume that will be part of a Volume Group and that VG will contain logical volumes. Start by running `pvscan` then `vgchange -ay` to activate all LVs in all VGs. So long as the VG and LV names do not conflict with the ones you already have present on the new machine then they should activate. Now you can mount those using the entries in /dev/mapper

Re: How to recover data from RAID1

Posted: 2021/09/14 19:45:22
by vicn1222
Thank you.

Done that too without success:

Code: Select all

[root@t320 archive]# pvscan
  PV /dev/sdc3   VG cs_t610         lvm2 [<1.82 TiB / 0    free]
  Total: 1 [<1.82 TiB] / in use: 1 [<1.82 TiB] / in no VG: 0 [0   ]
[root@t320 archive]# vgchange -ay
  2 logical volume(s) in volume group "cs_t610" now active
[root@t320 archive]# ls -l /dev/cs_t610/
total 0
lrwxrwxrwx. 1 root root 7 Sep 14 12:52 root -> ../dm-1
lrwxrwxrwx. 1 root root 7 Sep 14 12:52 swap -> ../dm-0
[root@t320 archive]# mount /dev/cs_t610/root /mnt/lideng/
mount: /mnt/lideng: mount(2) system call failed: Structure needs cleaning.
[root@t320 archive]# 

Re: How to recover data from RAID1

Posted: 2021/09/14 21:18:36
by TrevorH
mount: /mnt/lideng: mount(2) system call failed: Structure needs cleaning.
Sounds like an xfs filesystem? Try xfs_repair though you might want to take a bit for bit backup first if you have the space available.

Re: How to recover data from RAID1

Posted: 2021/09/15 02:23:49
by vicn1222
TrevorH wrote:
2021/09/14 21:18:36
Sounds like an xfs filesystem? Try xfs_repair though you might want to take a bit for bit backup first if you have the space available.
Hi TrevorH,

Thank you for help. Can I do bit for bit backup from 2T HD into 1T HD? There are only a few hundred GB data in the 2T HD.

Right now, I create a backup directory in the 1T HD, then doing below:

Code: Select all

cd backup
dd if=/dev/sdc of=./t610-backup.img bs=1M
It is still copying the file. If dd backup the entire 2T HD, then it won't fit the size....

Re: How to recover data from RAID1

Posted: 2021/09/15 05:34:54
by vicn1222
I did the xfs_repair and it mount now. All files are restored.

Thank you TrevorH!

Re: How to recover data from RAID1

Posted: 2021/09/15 18:13:44
by BShT
now backup it at regular basis, rasomwares are in the wild

Re: How to recover data from RAID1

Posted: 2021/09/15 19:17:55
by vicn1222
BShT wrote:
2021/09/15 18:13:44
now backup it at regular basis, rasomwares are in the wild
I am switching back to CentOS 7 for PowerEdge T610 machine. Some of its drivers are removed from CentOS 8. It will break if there is a kernel update.

I know this machine is old, but I can't just throw away this machine. It still runs perfectly if those drivers are not removed from CentOS 8.

Whoever made decision to remove the drivers is not a good person. Why waste other people's machine / money? Including those drivers is just a matter of recompiling the code, why remove? There won't be a need for dud (https://elrepo.org/linux/dud/el8/x86_64/) if they don't remove the drivers, which tells there must still be a big demand for old machine support.

Re: How to recover data from RAID1

Posted: 2021/09/16 12:49:36
by BShT
code needs to be maintained

it is not just a question to compile

Re: How to recover data from RAID1

Posted: 2021/09/17 02:48:31
by vicn1222
Don't see any maintenance cost.

Life has ended for both hardware and software. Nothing will be changed. Being a software developer myself, I am certainly sure for this.

The only cost is a few MB disk space to hold the source codes (cost for a few MB disk space is almost zero), and when there is a kernel or compiler update, a simple "gmake" to recompile the driver, which will be done on other drivers anyway.

In the last 20 years, there is not really a huger breakthrough on computer hardware. My T610 can take up to 192GB memory, and many T's hard driver with RAID 1 config. It is as good as many current servers, if not better. You can't just decide for me that I need to throw away such a great machine, simply because it is old. It is like forcing a 60 years old man to divorce his life long wife and marry to a 20 years old girl. And really, now a day, 60 years old are not really "old".