Centos75 kernel-3.10.0-862 impact CPU max MHz on Intel E5-2620v4
Centos75 kernel-3.10.0-862 impact CPU max MHz on Intel E5-2620v4
We encountered a critical issue when upgrade kernel from 3.10.0-693.17.1.el7 to kernel-3.10.0-862.14.4.el7 on a server with CPU E5-2620 v4.
After upgrade, the CPU max MHz was changed from 3000 to 2100 and system performance downgrade so much.
Attempt the following :
All kernel running with performance governor.
cpupower frequency-set -u 3000MHz
OR
/etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="intel_pstate=disable acpi=force”
OR
cat /sys/devices/system/cpu/intel_pstate/no_turbo
1
Enabled Intel Turbo Boost
But looks not take effect for changing.
kernel-3.10.0-693
Model name: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz
Stepping: 1
CPU MHz: 2099.752
CPU max MHz: 3000.0000
CPU min MHz: 1200.0000
kernel-3.10.0-862
Model name: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz
Stepping: 1
CPU MHz: 2099.615
CPU max MHz: 2100.0000
CPU min MHz: 1200.0000
I tested on the following kernels, and only all 862 kernel will change the max MHZ.
3.10.0-693.21.1.el7 3.10.0-693.17.1.el7 3.10.0-862.el7.x86_64 3.10.0-862.el7.x86_64 kernel-3.10.0-862.14.4.el7.x86_64
In addition, tested on another CPU, no impact on CPU E5-2667 v4. I installed kernel 693 and upgrade to 862, there is no change on max MHZ.
Model name: Intel(R) Xeon(R) CPU E5-2667 v4 @ 3.20GHz
Stepping: 1
CPU MHz: 1203.320
CPU max MHz: 3600.0000
CPU min MHz: 1200.0000
Just wondering if there is some document explaining the issue, or is there a way we can adjust the CPU max MHZ after upgraded to 3.10.0-862 kernel ?
Appreciate for any help.
Thanks
After upgrade, the CPU max MHz was changed from 3000 to 2100 and system performance downgrade so much.
Attempt the following :
All kernel running with performance governor.
cpupower frequency-set -u 3000MHz
OR
/etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="intel_pstate=disable acpi=force”
OR
cat /sys/devices/system/cpu/intel_pstate/no_turbo
1
Enabled Intel Turbo Boost
But looks not take effect for changing.
kernel-3.10.0-693
Model name: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz
Stepping: 1
CPU MHz: 2099.752
CPU max MHz: 3000.0000
CPU min MHz: 1200.0000
kernel-3.10.0-862
Model name: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz
Stepping: 1
CPU MHz: 2099.615
CPU max MHz: 2100.0000
CPU min MHz: 1200.0000
I tested on the following kernels, and only all 862 kernel will change the max MHZ.
3.10.0-693.21.1.el7 3.10.0-693.17.1.el7 3.10.0-862.el7.x86_64 3.10.0-862.el7.x86_64 kernel-3.10.0-862.14.4.el7.x86_64
In addition, tested on another CPU, no impact on CPU E5-2667 v4. I installed kernel 693 and upgrade to 862, there is no change on max MHZ.
Model name: Intel(R) Xeon(R) CPU E5-2667 v4 @ 3.20GHz
Stepping: 1
CPU MHz: 1203.320
CPU max MHz: 3600.0000
CPU min MHz: 1200.0000
Just wondering if there is some document explaining the issue, or is there a way we can adjust the CPU max MHZ after upgraded to 3.10.0-862 kernel ?
Appreciate for any help.
Thanks
Re: Centos75 kernel-3.10.0-862 impact CPU max MHz on Intel E5-2620v4
Try changing your BIOS settings for power management to 'performance' or 'maximum' or whatever your BIOS vendor terms it.
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke
Re: Centos75 kernel-3.10.0-862 impact CPU max MHz on Intel E5-2620v4
Thanks a lot, I'll take a try.
But a bit confused why it is not appeared in kernel 3.10.0-693*, is there any changed on kernel 893 about that I can search on?
But a bit confused why it is not appeared in kernel 3.10.0-693*, is there any changed on kernel 893 about that I can search on?
Re: Centos75 kernel-3.10.0-862 impact CPU max MHz on Intel E5-2620v4
That is a good question, but seems like a waste of time. You should first check with kernel-3.10.0-957.1.3.el7.x86_64 for it is the current kernel version.
Re: Centos75 kernel-3.10.0-862 impact CPU max MHz on Intel E5-2620v4
Thanks Jiehtone.
I tried the kernel 3.10.0-957.1.3.el7.x86_64 and seems same with kernel-3.10.0-862.
I tried the kernel 3.10.0-957.1.3.el7.x86_64 and seems same with kernel-3.10.0-862.
Re: Centos75 kernel-3.10.0-862 impact CPU max MHz on Intel E5-2620v4
Looks the kernel in centos7_5 and centos7_6 always can limit the CPU max MHz without Turbo mode .
And I tested in BIOS, we have to forcely set the Turbo mode enabled. OS/Kernel doesn't seem to be able to adjust intel_pstate as needed.
Not quite sure why this happened since centos7_5?
But thanks for your suggestion TrevorH and Jlehtone
And I tested in BIOS, we have to forcely set the Turbo mode enabled. OS/Kernel doesn't seem to be able to adjust intel_pstate as needed.
Not quite sure why this happened since centos7_5?
But thanks for your suggestion TrevorH and Jlehtone
Re: Centos75 kernel-3.10.0-862 impact CPU max MHz on Intel E5-2620v4
After enabled Turbo mode in BIOS, I just did sysbench test again on the following three of kernels.
3.10.0-693.21.1.el7.x86_64
3.10.0-862.14.4.el7.x86_64
3.10.0-957.1.3.el7.x86_64
Now, the CPU max MHz/cpu feq range on the three kernels are same.
Model name: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz
Stepping: 1
CPU MHz: 2299.992
CPU max MHz: 3000.0000
CPU min MHz: 1200.0000
analyzing CPU 0:
driver: intel_pstate
CPUs which run at the same hardware frequency: 0
CPUs which need to have their frequency coordinated by software: 0
maximum transition latency: Cannot determine or is not supported.
hardware limits: 1.20 GHz - 3.00 GHz
available cpufreq governors: performance powersave
current policy: frequency should be within 1.20 GHz and 3.00 GHz.
The governor "performance" may decide which speed to use
within this range.
current CPU frequency: 2.30 GHz (asserted by call to hardware)
boost state support:
Supported: yes
Active: yes
BUT, no lucks, 3.10.0-862 and 3.10.0-957 still quit slower than 3.10.0-693.21.1.el7.x86_64.
Could anyone help on other methods we can attempt to adjust?
3.10.0-693.21.1.el7.x86_64
3.10.0-862.14.4.el7.x86_64
3.10.0-957.1.3.el7.x86_64
Now, the CPU max MHz/cpu feq range on the three kernels are same.
Model name: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz
Stepping: 1
CPU MHz: 2299.992
CPU max MHz: 3000.0000
CPU min MHz: 1200.0000
analyzing CPU 0:
driver: intel_pstate
CPUs which run at the same hardware frequency: 0
CPUs which need to have their frequency coordinated by software: 0
maximum transition latency: Cannot determine or is not supported.
hardware limits: 1.20 GHz - 3.00 GHz
available cpufreq governors: performance powersave
current policy: frequency should be within 1.20 GHz and 3.00 GHz.
The governor "performance" may decide which speed to use
within this range.
current CPU frequency: 2.30 GHz (asserted by call to hardware)
boost state support:
Supported: yes
Active: yes
BUT, no lucks, 3.10.0-862 and 3.10.0-957 still quit slower than 3.10.0-693.21.1.el7.x86_64.
Could anyone help on other methods we can attempt to adjust?
Re: Centos75 kernel-3.10.0-862 impact CPU max MHz on Intel E5-2620v4
Have a look on:
Code: Select all
rpm -q --changelog kernel-3.10.0-862.14.4.el7.x86_64 | less
and it is not even the only such commit between 3.10.0-693.21.1 and 3.10.0-862.
Re: Centos75 kernel-3.10.0-862 impact CPU max MHz on Intel E5-2620v4
Ah, seem I find the reason why 3.10.0-862 and 3.10.0-957 quit slower than 3.10.0-693.21.1.el7.x86_64 on my test.
It is the case that kernel set my NIC parameter "busy-poll" to OFF
Kernel 7_5 7_6:
Kernel 7_4:
Because I'm using network-latency profile, and there are two setting related with
If I remove them in profile or change the both to 0, the performance of sysbench test will clearly be back to normal.
So for now, at least I have had a way to adjust the performance on new kernel(7_5/7_6).
But still confused the following:
why kernel change busy-poll from on to off and how can enable busy-poll on new kernel(7_5/7_6) ?
Appreciate for any help !!
It is the case that kernel set my NIC parameter "busy-poll" to OFF
Kernel 7_5 7_6:
Code: Select all
# ethtool -k eno2 | grep "busy-poll"
busy-poll: off [fixed]
Kernel 7_4:
Code: Select all
# ethtool -k eno2 | grep "busy-poll"
busy-poll: on [fixed]
Because I'm using network-latency profile, and there are two setting related with
Code: Select all
net.core.busy_read=50
net.core.busy_poll=50
So for now, at least I have had a way to adjust the performance on new kernel(7_5/7_6).
But still confused the following:
why kernel change busy-poll from on to off and how can enable busy-poll on new kernel(7_5/7_6) ?
Appreciate for any help !!
Re: Centos75 kernel-3.10.0-862 impact CPU max MHz on Intel E5-2620v4
You declare that minimal network latency is most important to you and then you benchmark something else and complain?I'm using network-latency profile