Building RAID6, very low speed

Issues related to applications and software problems
Post Reply
caphix
Posts: 7
Joined: 2020/02/12 21:13:29

Building RAID6, very low speed

Post by caphix » 2020/02/12 21:21:43

Hi, I am building a RAID 6 on 5 HDDs 4TB each.

mdadm --create /dev/md0 --level=6 --raid-devices=5 /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1 –verbose

the RAID6 starts with speed very low. then i tune it with:

# echo 100000 > /proc/sys/dev/raid/speed_limit_min
# echo 5000000 > /proc/sys/dev/raid/speed_limit_max

the speed goes up but then slows down. I have started the RAID on the 6 of February and today after a week it is sill

active raid6 sdg[3] sdf[2] sdd[1] sdc[0]
7813772288 blocks super 1.2 level 6, 512k chunk, algorithm 2 [5/5] [UUUUU]
[=======>.............] resync = 38.6% (1510415736/3906886144) finish=18792.7min speed=2125K/sec
bitmap: 19/30 pages [76KB], 65536KB chunk


What do I do wrong? is it normal that to build a RAID will take so long? weeks

Thanks so much

User avatar
TrevorH
Forum Moderator
Posts: 29112
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: Building RAID6, very low speed

Post by TrevorH » 2020/02/12 23:10:05

Are your partitions correctly aligned?
CentOS 6 will die in November 2020 - migrate sooner rather than later!
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 is dead, do not use it.
Full time Geek, part time moderator. Use the FAQ Luke

caphix
Posts: 7
Joined: 2020/02/12 21:13:29

Re: Building RAID6, very low speed

Post by caphix » 2020/02/13 07:43:47

TrevorH wrote:
2020/02/12 23:10:05
Are your partitions correctly aligned?
Thanks TrevorH. Sorry my ignorance, what does partitions correctly aligned? How do you do that? Do I have to do it before I run mdadm --create?
Thanks

User avatar
TrevorH
Forum Moderator
Posts: 29112
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: Building RAID6, very low speed

Post by TrevorH » 2020/02/13 09:57:26

Most modern disks have 4KB sectors so it is important that all partitions are aligned on boundaries that are exactly divisible by 4096. The usual recommendation is to make partitions start on 1MB boundaries. Use fdisk -lu to see what starting sectors your partitions use then divide those numbers by 8 to see if they are correct.
CentOS 6 will die in November 2020 - migrate sooner rather than later!
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 is dead, do not use it.
Full time Geek, part time moderator. Use the FAQ Luke

caphix
Posts: 7
Joined: 2020/02/12 21:13:29

Re: Building RAID6, very low speed

Post by caphix » 2020/02/13 10:26:52

thanks TrevorH
TrevorH wrote:
2020/02/13 09:57:26
Most modern disks have 4KB sectors so it is important that all partitions are aligned on boundaries that are exactly divisible by 4096. The usual recommendation is to make partitions start on 1MB boundaries. Use fdisk -lu to see what starting sectors your partitions use then divide those numbers by 8 to see if they are correct.
this is what it shows:
root# fdisk -lu /dev/sdb
The primary GPT table is corrupt, but the backup appears OK, so that will be used.
Disk /dev/sdb: 3.7 TiB, 4000787030016 bytes, 7814037168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 0140D4FD-C537-446A-BFCD-E4B0822B412C

Device Start End Sectors Size Type
/dev/sdb1 2048 7814035455 7814033408 3.7T Linux RAID

root:~# fdisk -lu /dev/sde
Disk /dev/sde: 3.7 TiB, 4000787030016 bytes, 7814037168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

root~# fdisk -lu /dev/sdg
Disk /dev/sdg: 3.7 TiB, 4000787030016 bytes, 7814037168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

root:~# fdisk -lu /dev/sdf
Disk /dev/sdf: 3.7 TiB, 4000787030016 bytes, 7814037168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
The Raid 6 is still running. Can I align the disks while the raid is running?
Do I need to do the alignment for all the disks or just for the first one?
thanks

tunk
Posts: 725
Joined: 2017/02/22 15:08:17

Re: Building RAID6, very low speed

Post by tunk » 2020/02/13 10:43:18

I have no experience with software RAIDs, but it looks
like you have no partitions on sde, sdg and sdf.

caphix
Posts: 7
Joined: 2020/02/12 21:13:29

Re: Building RAID6, very low speed

Post by caphix » 2020/02/13 10:51:32

tunk wrote:
2020/02/13 10:43:18
I have no experience with software RAIDs, but it looks
like you have no partitions on sde, sdg and sdf.
Thanks tunk.

So probably I need to partition all my 5 drives before starting the raid?
It's my first time with software raid too.

hunter86_bg
Posts: 2016
Joined: 2015/02/17 15:14:33
Location: Bulgaria
Contact:

Re: Building RAID6, very low speed

Post by hunter86_bg » 2020/02/14 14:23:11

If the software raid was in sync - you could do it 1 by 1 .
In your case it will take ages - so you have to start over.
Create the partitions and then use them for software raid.

Also you have another option - you can use the disks as PVs and then create LVs of type raid6. This will allow you in the future to reshape the LVs to any kind of raid.

User avatar
TrevorH
Forum Moderator
Posts: 29112
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: Building RAID6, very low speed

Post by TrevorH » 2020/02/14 14:27:30

I do not think the lack of partitions is the source of the slowness. It's fine to use mdadm on whole drives as long as you and any other admin that might come along understands that the disks are part of an array and don't look at them and go "oh, no partition table, that can't be in use, let me format it".
CentOS 6 will die in November 2020 - migrate sooner rather than later!
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 is dead, do not use it.
Full time Geek, part time moderator. Use the FAQ Luke

Post Reply

Return to “CentOS 7 - Software Support”