Cloning a NAS hard drive with OS

Issues related to hardware problems
mjz
Posts: 115
Joined: 2020/02/13 22:05:02

Cloning a NAS hard drive with OS

Post by mjz » 2023/01/16 19:40:27

I have a 4TB NAS drive with OS installed in a network with my Centos server.
The drive is going bad (reallocated sectors popping up) so I decided to clone it to a new identical drive.

I started a dd if=old drive of=new drive with bs=1M and noticed it is transferring at 15.1 MB/sec. At this rate it will take 3 days to copy over!
Is this to be expected?

Is there a better and faster way to do this?
sdb is the old drive & sdc is the new one.
sdb 8:16 0 3.6T 0 disk
├─sdb1 8:17 0 2G 0 part
│ └─md127 9:127 0 2G 0 raid1
├─sdb2 8:18 0 3.6T 0 part
├─sdb3 8:19 0 1G 0 part
├─sdb4 8:20 0 1G 0 part
├─sdb5 8:21 0 1G 0 part
├─sdb6 8:22 0 2G 0 part
└─sdb7 8:23 0 1G 0 part
sdc 8:32 0 3.6T 0 disk
Should lsblk shows the above after 3 hours running. I thought I would see the partitions in sdc?

User avatar
jlehtone
Posts: 4523
Joined: 2007/12/11 08:17:33
Location: Finland

Re: Cloning a NAS hard drive with OS

Post by jlehtone » 2023/01/16 20:49:21

The kernel does read partition tables to memory during boot.

Partition tools (fdisk, gdisk, parted) occasionally do note that
"kernel is still using old values" when you save changes to partition table.
The partition tools at least attempt to notify the kernel. The dd does not.

Therefore, the 'lsblk' most likely shows values from memory.

User avatar
TrevorH
Site Admin
Posts: 33202
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: Cloning a NAS hard drive with OS

Post by TrevorH » 2023/01/16 21:44:41

Use status=progress on dd along with oflag=direct and it will report actual disk transfer rates.

I presume you have removed this drive from the system and it is not mounted. If not then it is not safe to copy until nothing is using it.

It may be that your source drive is the problem, not the target. You could try reading from the source and writing to /dev/null to see if it is getting so many I/O errors that it's very slow to read from.
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

mjz
Posts: 115
Joined: 2020/02/13 22:05:02

Re: Cloning a NAS hard drive with OS

Post by mjz » 2023/01/16 21:57:48

It is the source drive which is why I am cloning to a new drive. A few hundred sectors have been reallocated.

Both drives, of course are unmounted. Status = progress shows:
9247391744 bytes (9.2 GB, 8.6 GiB) copied, 632.323 s, 14.6 MB/s
9248440320 bytes (9.2 GB, 8.6 GiB) copied, 651 s, 14.2 MB/s
dd: error reading '/dev/sdb': Input/output error
8809+11 records in
8820+0 records out
9248440320 bytes (9.2 GB, 8.6 GiB) copied, 653.879 s, 14.1 MB/s
219084226560 bytes (219 GB, 204 GiB) copied, 14171 s, 15.5 MB/s
The input/output error report occurred in the first few minutes of the dd transfer. I just wonder if the "error" will negate the entire cloning process. At the rate shown above it is going to take 3 days to clone this drive.

I used block size = 1M.

User avatar
TrevorH
Site Admin
Posts: 33202
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: Cloning a NAS hard drive with OS

Post by TrevorH » 2023/01/16 22:18:11

I'd suggest installing and using dd_rescue from EPEL and using that. It tries harder and writes what it can read to the output, saving as much as it can.
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

mjz
Posts: 115
Joined: 2020/02/13 22:05:02

Re: Cloning a NAS hard drive with OS

Post by mjz » 2023/01/17 12:59:10

I have ddrescue running (started last night).
So far this morning after 12 hours it has read 670 GB out of the 4TB disk. 17% rescued.
current transfer speed is ~15 MB/second.
remaining time = 2 days 12 hours left to go.
Read errors = 2 (which occurred immediately when ddrescue started). No errors since.

logfile shows:
# Mapfile. Created by GNU ddrescue version 1.23
# Command line: ddrescue -f -n -r3 /dev/sdb /dev/sdc /root/ddrescue.log
# Start time: 2023-01-16 18:07:51
# Current time: 2023-01-17 06:51:31
# Copying non-tried blocks... Pass 1 (forwards)
# current_pos current_status current_pass
0x9AC79F0000 ? 1
# pos size status
0x00000000 0x17F9A6000 +
0x17F9A6000 0x0000A000 *
0x17F9B0000 0x02630000 ?
0x181FE0000 0xA4C23000 +
0x226C03000 0x0000D000 *
0x226C10000 0x02630000 ?
0x229240000 0x989E7C0000 +
0x9AC7A00000 0x308B9DD6000 ?
This is a painfully slow process,
I assume this is normal and that recovery is occurring.

User avatar
TrevorH
Site Admin
Posts: 33202
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: Cloning a NAS hard drive with OS

Post by TrevorH » 2023/01/17 14:27:44

At least it's working for some value of working! I suspect the issue is that when it tries to read certain sectors, the drive itself does a lot of retries before it reports back to the operating system "sorry that didn't work". But if you want the maximum amount of data possible then you probably have to just live with teh speed and hope that as it gets further through the disk the problems get fewer not more widespread.
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

mjz
Posts: 115
Joined: 2020/02/13 22:05:02

Re: Cloning a NAS hard drive with OS

Post by mjz » 2023/01/17 14:48:55

Trevor - thanks for your suggestion. 'ddrescue' is more informative than "dd" and does seem to have better error recovery. Since my last post, the disk is still showing a couple of read errors and a steady 15 MB/sec transfer. I am hoping that since the reallocation of bad sectors was just beginning to happen on the old drive and the disk was running fine (NAS disk), this "rescue" will be successful.

Question: Once the bit for bit copy is complete, is my new drive essentially a perfect copy of the old one including bad/reallocated sectors? I have not done this before. I have always just junked the old failing drive and used backup to restore files (tried and true method!). My expectation is I can just take the new drive, reconnect it to the processor in the NAS box and it should just boot up and work? Everything on my Centos server should access it the same as before (i.e. same UUID, etc.).

User avatar
TrevorH
Site Admin
Posts: 33202
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: Cloning a NAS hard drive with OS

Post by TrevorH » 2023/01/17 18:26:17

I don't know the answer. I would guess it might depend on the NAS manufacturer and how they've set it up but hopefully it's transplantable - they probably clone them in the factory rather than go through an install.

I think dd_rescue writes a sector of binary zeroes if the origin could not be read. And re-allocated blocks are hardware, internal to the drive, so they are entirely transparent at any layer above that. You'll almost certainly want to get the NAS to run its equivalent of fsck once you do get it booted. Let us know how it worked in a few days!
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

mjz
Posts: 115
Joined: 2020/02/13 22:05:02

Re: Cloning a NAS hard drive with OS

Post by mjz » 2023/01/17 20:29:39

I seem to recall this NAS system did this when booting up after a power failure and the backup battery ran down.

e2fsck -pf /dev/sda

I may do this after the drive is completely cloned.

Post Reply