Page 1 of 1
Building RAID6, very low speed
Posted: 2020/02/12 21:21:43
by caphix
Hi, I am building a RAID 6 on 5 HDDs 4TB each.
mdadm --create /dev/md0 --level=6 --raid-devices=5 /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1 –verbose
the RAID6 starts with speed very low. then i tune it with:
# echo 100000 > /proc/sys/dev/raid/speed_limit_min
# echo 5000000 > /proc/sys/dev/raid/speed_limit_max
the speed goes up but then slows down. I have started the RAID on the 6 of February and today after a week it is sill
active raid6 sdg[3] sdf[2] sdd[1] sdc[0]
7813772288 blocks super 1.2 level 6, 512k chunk, algorithm 2 [5/5] [UUUUU]
[=======>.............] resync = 38.6% (1510415736/3906886144) finish=18792.7min speed=2125K/sec
bitmap: 19/30 pages [76KB], 65536KB chunk
What do I do wrong? is it normal that to build a RAID will take so long? weeks
Thanks so much
Re: Building RAID6, very low speed
Posted: 2020/02/12 23:10:05
by TrevorH
Are your partitions correctly aligned?
Re: Building RAID6, very low speed
Posted: 2020/02/13 07:43:47
by caphix
TrevorH wrote: ↑2020/02/12 23:10:05
Are your partitions correctly aligned?
Thanks TrevorH. Sorry my ignorance, what does partitions correctly aligned? How do you do that? Do I have to do it before I run mdadm --create?
Thanks
Re: Building RAID6, very low speed
Posted: 2020/02/13 09:57:26
by TrevorH
Most modern disks have 4KB sectors so it is important that all partitions are aligned on boundaries that are exactly divisible by 4096. The usual recommendation is to make partitions start on 1MB boundaries. Use fdisk -lu to see what starting sectors your partitions use then divide those numbers by 8 to see if they are correct.
Re: Building RAID6, very low speed
Posted: 2020/02/13 10:26:52
by caphix
thanks TrevorH
TrevorH wrote: ↑2020/02/13 09:57:26
Most modern disks have 4KB sectors so it is important that all partitions are aligned on boundaries that are exactly divisible by 4096. The usual recommendation is to make partitions start on 1MB boundaries. Use fdisk -lu to see what starting sectors your partitions use then divide those numbers by 8 to see if they are correct.
this is what it shows:
root# fdisk -lu /dev/sdb
The primary GPT table is corrupt, but the backup appears OK, so that will be used.
Disk /dev/sdb: 3.7 TiB, 4000787030016 bytes, 7814037168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 0140D4FD-C537-446A-BFCD-E4B0822B412C
Device Start End Sectors Size Type
/dev/sdb1 2048 7814035455 7814033408 3.7T Linux RAID
root:~# fdisk -lu /dev/sde
Disk /dev/sde: 3.7 TiB, 4000787030016 bytes, 7814037168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
root~# fdisk -lu /dev/sdg
Disk /dev/sdg: 3.7 TiB, 4000787030016 bytes, 7814037168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
root:~# fdisk -lu /dev/sdf
Disk /dev/sdf: 3.7 TiB, 4000787030016 bytes, 7814037168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
The Raid 6 is still running. Can I align the disks while the raid is running?
Do I need to do the alignment for all the disks or just for the first one?
thanks
Re: Building RAID6, very low speed
Posted: 2020/02/13 10:43:18
by tunk
I have no experience with software RAIDs, but it looks
like you have no partitions on sde, sdg and sdf.
Re: Building RAID6, very low speed
Posted: 2020/02/13 10:51:32
by caphix
tunk wrote: ↑2020/02/13 10:43:18
I have no experience with software RAIDs, but it looks
like you have no partitions on sde, sdg and sdf.
Thanks tunk.
So probably I need to partition all my 5 drives before starting the raid?
It's my first time with software raid too.
Re: Building RAID6, very low speed
Posted: 2020/02/14 14:23:11
by hunter86_bg
If the software raid was in sync - you could do it 1 by 1 .
In your case it will take ages - so you have to start over.
Create the partitions and then use them for software raid.
Also you have another option - you can use the disks as PVs and then create LVs of type raid6. This will allow you in the future to reshape the LVs to any kind of raid.
Re: Building RAID6, very low speed
Posted: 2020/02/14 14:27:30
by TrevorH
I do not think the lack of partitions is the source of the slowness. It's fine to use mdadm on whole drives as long as you and any other admin that might come along understands that the disks are part of an array and don't look at them and go "oh, no partition table, that can't be in use, let me format it".