CentOS

Hardware:
Supermicro SYS-5019C-M
Xeon E-2174G CPU
32G RAM

We have been working on a home brewed NAS with hot swap drives. The first OS we tried was FreeNAS. Good OS overall however it really has trouble with hot swapping drives. Excellent throughput via NFS. Note that both servers are connected via 10 Gigabit Ethernet links.

Because of the problem with drive swaps we switched to CentOS. Much better handling of drive swaps however we're maxing out at 40Mbps with an average 6MBps. This is terrible for a 10G link.

I know through testing that a Windows to Windows setup can get around 400MBps over this same infrastructure and again the FreeNAS setup achieved much higher throughput also over the same infrastructure. What am I missing in configuring CentOS for 10G Ethernet?

jumbo frames?

i know nothing about your HD interface but you have to give raw HDs to ZFS.

Sorry. These are SATA drives with EXT4. This shouldn't be limited to an average of 6MBps.

As for the jumbo frames as I mentioned earlier I can reliably get a Windows server to hit 400MBps via this infrastructure without network tweaking. I should at least be seeing 100MBps or more I would think.

What is the output from lspci -nn | grep -i net and from uname -a

# lspci -nn | grep -i net
01:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 [8086:1528] (rev 01)
03:00.0 Ethernet controller [0200]: Intel Corporation I210 Gigabit Network Connection [8086:1533] (rev 03)
04:00.0 Ethernet controller [0200]: Intel Corporation I210 Gigabit Network Connection [8086:1533] (rev 03)
# uname -a
Linux Server.company.net 4.18.0-193.14.2.el8_2.x86_64 #1 SMP Sun Jul 26 03:54:29 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux