I have a hyper-v 2012 R2 cluster running multiple VM's with good redundant networking throughout. There are other HAProxy VM's on this cluster which have no issues, these are our first pair of CentOS 8 HAProxies.
The deployment is fairly typical with two identical VM's running HAProxy and keepalived. The issue occurs on either VM in the pair.
Internet --> Firewall (checkpoint) -Network1-> These CentOS 8 VM's -LB Network-> Backend Servers
The backend servers are running Exchange 2016 on Windows Server 2016
Symptom - after a period of time (around 2-3 weeks) the VM will start getting intermittent network connectivity to the hosts it is load balancing
I have one of these CentOS 8 vm's exhibiting the issue at the moment. It seems if I fail this over (live migrate) to another node then the issue goes away.
I have left a constant ping running from the CentOS 8 VM to one of the backend servers and see no dropped pings.
If I manually try to curl one of the exchange servers around one in eight requests just times out.
I've tried running a tcpdump while the issue is happening, with a filter that I believe should show syn requests:
Code: Select all
tcpdump -i eth1 -nnSX "tcp[tcpflags] & (tcp-syn) != 0 && port 80"
I'm seeing nothing untoward in /var/log/messages
Any pointers would be greatly appreciated before I resort to rebuilding with CentOS 7