1Gb NIC problems

Issues related to configuring your network
bore
Posts: 7
Joined: 2007/06/20 08:49:46

1Gb NIC problems

Post by bore » 2007/06/20 08:59:08

Hi all,
I am trying to run an 1Gb network card on my CentOS 4 and I am stuck
Here is the situation
2 PCs, 2 network cards, with Realtek 8169 chips and CentOS 4 (2.6.9-22.EL) and cross cable between them
Installation goes fine,cards are recognized, ethtoll says 1000Mb, full duplex, TP port BUT when I try copy a file from one PC to another, the transfer does not goes over 11MB/sec which is under 100Mb. Exactly same transfer I get over the other 100 Mb network card.
I thought the disks are slow, so I made ram disks and now I copy a file from a ram disk of PC A to the ram disk of PC B, using scp command, but still transfer remains the same
Downloaded latest driver from Realtek, installed...still the same.

I am out of ideas
Any help please.

Thanks

kentyler
Posts: 161
Joined: 2007/06/20 13:31:05
Location: Northeast Ohio
Contact:

Re: 1Gb NIC problems

Post by kentyler » 2007/06/20 15:13:52

Your problem may be hostbound.

When you do this transfer what does this line in top say?

Cpu(s): 50.0% us, 6.6% sy, 0.0% ni, 0.0% id, 41.7% wa, 1.0% hi, 0.7% si


Trying to do the transfer across the network, and also scp to localhost and put the file in a different directory.

When you do ifconfig do you see anything but a 0 in the following?

RX errors:0 dropped:0 overruns:0 frame:0
TX errors:0 dropped:0 overruns:0 carrier:0
collisions:0

On an AMD 1ghz with ide hd I get the following running the local test;

[root@dmzdev ~]# scp /test.tgz localhost:/root
root@localhost's password:
test.tgz 100% 260MB 5.8MB/s 00:45

michaelnel
Posts: 1478
Joined: 2006/05/29 16:50:11
Location: San Francisco, CA

Re: 1Gb NIC problems

Post by michaelnel » 2007/06/20 17:26:50

Try using ttcp to test the speed.

User avatar
WhatsHisName
Posts: 1547
Joined: 2005/12/19 20:21:43
Location: /earth/usa/nj

Re: 1Gb NIC problems

Post by WhatsHisName » 2007/06/20 20:40:15

michaelnel: Thanks for the pointer about ttcp. That’s a really nice memory-to-memory testing utility.

On my copper gigabit network, ttcp confirmed things that I already knew (i.e., that typical large transfers run in the 200-300Mbps range), but was shocked to see that two modern systems were swapping spit at 850-900Mbps.

michaelnel
Posts: 1478
Joined: 2006/05/29 16:50:11
Location: San Francisco, CA

Re: 1Gb NIC problems

Post by michaelnel » 2007/06/20 21:25:26

We used to use ttcp to test wire speed when I was in support at Silicon Graphics. You can isolate the network from other issues such as disk i/o (but you can also feed files through it). You can vary block sizes and other parameters too.

It's a reliable tool and runs on most unices and windows. Good one to have in the toolbox.

bore
Posts: 7
Joined: 2007/06/20 08:49:46

Re: 1Gb NIC problems

Post by bore » 2007/06/21 06:07:46

CPU 53.9% us, 24.3% sy, 1.6% ni, 9.2% id,0.0% wa, 10.9% hi 0.0% si

Thanks

bore
Posts: 7
Joined: 2007/06/20 08:49:46

Re: 1Gb NIC problems

Post by bore » 2007/06/21 08:16:32

Using ttcp transfer seems to be better - around 34M
Now the question is why SCP is not getting that speed

Thanks to all
Boris

bore
Posts: 7
Joined: 2007/06/20 08:49:46

Re: 1Gb NIC problems

Post by bore » 2007/06/21 09:07:55

Gyus
10x to everyone who replayed. Problem is clarified
ttcp is cool, although I had to install it additionally
Measuring with ttcp showed
ttcp-t: 8192000000 bytes in 163.574 real seconds = 48907.540 KB/sec +++
ttcp-t: 8192000000 bytes in 157.992 CPU seconds = 50635.481 KB/cpu sec
ttcp-t: 1000000 I/O calls, msec/call = 0.167, calls/sec = 6113.443

This is definitely Gigabith

Trough the tinme while ttcp wodks, top shows
us 5%
sy 45%
hi 50%
id 0%

So obviously CPU is the bottleneck

Please comment if I am wrong

Thanks again
Boris

michaelnel
Posts: 1478
Joined: 2006/05/29 16:50:11
Location: San Francisco, CA

Re: 1Gb NIC problems

Post by michaelnel » 2007/06/21 17:51:01

Are you using jumbo frames? They can considerably reduce cpu utilization.

EDIT: looks like your chipset has lame jf support, only up to 7.5K (full jf is 9.0K). I think I'd get a better NIC.

bore
Posts: 7
Joined: 2007/06/20 08:49:46

Re: 1Gb NIC problems

Post by bore » 2007/06/22 07:21:57

Thanks
I enabled Jumbo frames and now it runs significantly faster, however I faced the following problem
when I set the transmit site mtu to some value, the receive site mtu to the same value, the connection hangs
say if I set the receiving site to 7200, the transmit site to 7200 - hang
even if i set them to 3000 - hang
on 1500 work however
If I set one of the cards say to 7198 and the other to 7200 then it operates fine
Can u advise about that?
Also I used ifconfig mtu to enable jumbo frames
Can u asvise how to make that permanent trough restarts?

Thanks and regards
Boris

Post Reply

Return to “CentOS 4 - Networking Support”