ksoftirqd service utilized 100% of CPU

Issues related to configuring your network
Post Reply
apol
Posts: 2
Joined: 2021/09/17 01:25:38

ksoftirqd service utilized 100% of CPU

Post by apol » 2021/09/17 01:39:06

Hello everyone

we are having trouble with out database server running centos 7 and mariadb. The server is running in KVM and in bridge mode.here are some details about my vm

1. assigned 14 vCPUs
2. 64GB memory


Both servers are connecting in 1 smart switch but in their own vlan.

Lately, the "ksoftirqd" service of our vm dbserver is utilizing 100% of the server CPUs, affecting the server's performance.

when I checked the ifconfig, the TX collision value is more than 500T.

I tried to change the MTU value of the interface but editing the ifcfg-ens3. but to no avail, the MTU doesn't changed.

What seems to be causing the "ksoftirqd" to utilized the big part of the vCPU? 2T daily users are affected by this issue.


Appreciate your input and suggestions on how to resolve this issue of ours.

User avatar
TrevorH
Site Admin
Posts: 33191
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: ksoftirqd service utilized 100% of CPU

Post by TrevorH » 2021/09/17 08:26:26

Is this VM doing lots of I/O?

I've found that KVM VMs have a limitation - up to and including 8 vcpus, it distributes interrupts nicely across all 8 cores. Go to 9 cores and there's a magic change and all of a sudden, all IRQ interrupts are taken only on core 0. Do lots of I/O and generate lots of interrupts and sooner rather than later, core 0 runs out of oomph and cannot handle all those interrupts arriving at once. For my use case, we do lots of network traffic, smallish UDP packets and lots of them and for us, we cannot use a single VM to push much more than about 200-250Mbps before we run into a wall. We actually find that we can do more with an 8 core VM than we can with one with say 12 cores.

Check /proc/interrupts and see if this is happening to you.

There are some things you can do to try to help but check /proc/interrupts and see if you are affected like this.
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

apol
Posts: 2
Joined: 2021/09/17 01:25:38

Re: ksoftirqd service utilized 100% of CPU

Post by apol » 2021/09/18 11:10:41

thank you very much! I appreciate it. I'll try to check the interrupts

Post Reply