Centos75 kernel-3.10.0-862 impact CPU max MHz on Intel E5-2620v4

Issues related to hardware problems
jizh
Posts: 11
Joined: 2018/12/26 06:27:45

Centos75 kernel-3.10.0-862 impact CPU max MHz on Intel E5-2620v4

Post by jizh » 2018/12/26 07:15:49

We encountered a critical issue when upgrade kernel from 3.10.0-693.17.1.el7 to kernel-3.10.0-862.14.4.el7 on a server with CPU E5-2620 v4.
After upgrade, the CPU max MHz was changed from 3000 to 2100 and system performance downgrade so much.


Attempt the following :
All kernel running with performance governor.
cpupower frequency-set -u 3000MHz
OR
/etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="intel_pstate=disable acpi=force”
OR
cat /sys/devices/system/cpu/intel_pstate/no_turbo
1
Enabled Intel Turbo Boost
But looks not take effect for changing.



kernel-3.10.0-693
Model name: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz
Stepping: 1
CPU MHz: 2099.752
CPU max MHz: 3000.0000
CPU min MHz: 1200.0000



kernel-3.10.0-862
Model name: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz
Stepping: 1
CPU MHz: 2099.615
CPU max MHz: 2100.0000
CPU min MHz: 1200.0000


I tested on the following kernels, and only all 862 kernel will change the max MHZ.
3.10.0-693.21.1.el7 3.10.0-693.17.1.el7 3.10.0-862.el7.x86_64 3.10.0-862.el7.x86_64 kernel-3.10.0-862.14.4.el7.x86_64






In addition, tested on another CPU, no impact on CPU E5-2667 v4. I installed kernel 693 and upgrade to 862, there is no change on max MHZ.

Model name: Intel(R) Xeon(R) CPU E5-2667 v4 @ 3.20GHz
Stepping: 1
CPU MHz: 1203.320
CPU max MHz: 3600.0000
CPU min MHz: 1200.0000







Just wondering if there is some document explaining the issue, or is there a way we can adjust the CPU max MHZ after upgraded to 3.10.0-862 kernel ?

Appreciate for any help.
Thanks

User avatar
TrevorH
Forum Moderator
Posts: 29681
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: Centos75 kernel-3.10.0-862 impact CPU max MHz on Intel E5-2620v4

Post by TrevorH » 2018/12/26 14:53:37

Try changing your BIOS settings for power management to 'performance' or 'maximum' or whatever your BIOS vendor terms it.
CentOS 6 will die in November 2020 - migrate sooner rather than later!
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 is dead, do not use it.
Full time Geek, part time moderator. Use the FAQ Luke

jizh
Posts: 11
Joined: 2018/12/26 06:27:45

Re: Centos75 kernel-3.10.0-862 impact CPU max MHz on Intel E5-2620v4

Post by jizh » 2018/12/27 10:57:34

Thanks a lot, I'll take a try.
But a bit confused why it is not appeared in kernel 3.10.0-693*, is there any changed on kernel 893 about that I can search on?

User avatar
jlehtone
Posts: 3101
Joined: 2007/12/11 08:17:33
Location: Finland

Re: Centos75 kernel-3.10.0-862 impact CPU max MHz on Intel E5-2620v4

Post by jlehtone » 2018/12/27 11:27:31

That is a good question, but seems like a waste of time. You should first check with kernel-3.10.0-957.1.3.el7.x86_64 for it is the current kernel version.

jizh
Posts: 11
Joined: 2018/12/26 06:27:45

Re: Centos75 kernel-3.10.0-862 impact CPU max MHz on Intel E5-2620v4

Post by jizh » 2018/12/28 04:06:16

Thanks Jiehtone.

I tried the kernel 3.10.0-957.1.3.el7.x86_64 and seems same with kernel-3.10.0-862.

jizh
Posts: 11
Joined: 2018/12/26 06:27:45

Re: Centos75 kernel-3.10.0-862 impact CPU max MHz on Intel E5-2620v4

Post by jizh » 2018/12/28 04:23:28

Looks the kernel in centos7_5 and centos7_6 always can limit the CPU max MHz without Turbo mode .

And I tested in BIOS, we have to forcely set the Turbo mode enabled. OS/Kernel doesn't seem to be able to adjust intel_pstate as needed.


Not quite sure why this happened since centos7_5?


But thanks for your suggestion TrevorH and Jlehtone

jizh
Posts: 11
Joined: 2018/12/26 06:27:45

Re: Centos75 kernel-3.10.0-862 impact CPU max MHz on Intel E5-2620v4

Post by jizh » 2018/12/28 06:37:08

After enabled Turbo mode in BIOS, I just did sysbench test again on the following three of kernels.
3.10.0-693.21.1.el7.x86_64
3.10.0-862.14.4.el7.x86_64
3.10.0-957.1.3.el7.x86_64


Now, the CPU max MHz/cpu feq range on the three kernels are same.

Model name: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz
Stepping: 1
CPU MHz: 2299.992
CPU max MHz: 3000.0000
CPU min MHz: 1200.0000


analyzing CPU 0:
driver: intel_pstate
CPUs which run at the same hardware frequency: 0
CPUs which need to have their frequency coordinated by software: 0
maximum transition latency: Cannot determine or is not supported.
hardware limits: 1.20 GHz - 3.00 GHz
available cpufreq governors: performance powersave
current policy: frequency should be within 1.20 GHz and 3.00 GHz.
The governor "performance" may decide which speed to use
within this range.
current CPU frequency: 2.30 GHz (asserted by call to hardware)
boost state support:
Supported: yes
Active: yes


BUT, no lucks, 3.10.0-862 and 3.10.0-957 still quit slower than 3.10.0-693.21.1.el7.x86_64.

Could anyone help on other methods we can attempt to adjust?

User avatar
jlehtone
Posts: 3101
Joined: 2007/12/11 08:17:33
Location: Finland

Re: Centos75 kernel-3.10.0-862 impact CPU max MHz on Intel E5-2620v4

Post by jlehtone » 2018/12/28 09:32:48

jizh wrote:
2018/12/27 10:57:34
is there any changed on kernel 893 about that I can search on?
Have a look on:

Code: Select all

rpm -q --changelog kernel-3.10.0-862.14.4.el7.x86_64 | less
At least the Wed Sep 20 2017 [3.10.0-712.el7] commit is full of [cpufreq] intel_pstate notes
and it is not even the only such commit between 3.10.0-693.21.1 and 3.10.0-862.

jizh
Posts: 11
Joined: 2018/12/26 06:27:45

Re: Centos75 kernel-3.10.0-862 impact CPU max MHz on Intel E5-2620v4

Post by jizh » 2019/01/02 03:51:17

Ah, seem I find the reason why 3.10.0-862 and 3.10.0-957 quit slower than 3.10.0-693.21.1.el7.x86_64 on my test.

It is the case that kernel set my NIC parameter "busy-poll" to OFF
Kernel 7_5 7_6:

Code: Select all

# ethtool -k eno2 | grep "busy-poll"
busy-poll: off [fixed]


Kernel 7_4:

Code: Select all

# ethtool -k eno2 | grep "busy-poll"
busy-poll: on [fixed]

Because I'm using network-latency profile, and there are two setting related with

Code: Select all

net.core.busy_read=50
net.core.busy_poll=50
If I remove them in profile or change the both to 0, the performance of sysbench test will clearly be back to normal.


So for now, at least I have had a way to adjust the performance on new kernel(7_5/7_6).

But still confused the following:
why kernel change busy-poll from on to off and how can enable busy-poll on new kernel(7_5/7_6) ?



Appreciate for any help !!

chemal
Posts: 695
Joined: 2013/12/08 19:44:49

Re: Centos75 kernel-3.10.0-862 impact CPU max MHz on Intel E5-2620v4

Post by chemal » 2019/01/02 04:43:43

I'm using network-latency profile
You declare that minimal network latency is most important to you and then you benchmark something else and complain?

Post Reply

Return to “CentOS 7 - Hardware Support”