Network Policy enabling in CentOS7 node causes issues.

[monesh]

Hello everyone, I have an AWS EKS-1.25 cluster with CentOS7 worker nodes. Have installed Kernel version 5.10+. Then tried enabling the NetworkPolicy through AWS VPC CNI addon and could see few errors. I have tried the same steps with Linux and Ubuntu os, found no issues. So I thought of checking if there is anything to do specifically in CentOS7 to make it work. Any help would be appreciated.

In the worker nodes under /var/log/aws-routed-eni/network-policy-agent.log

{"level":"error","timestamp":"2023-11-08T14:51:35.572Z","logger":"ebpf-client","msg":"Unable to load events binary. Required for policy enforcement, exiting..","error":"failed to load maps","stacktrace":"github.com/aws/aws-network-policy-agent/pkg/ebpf.NewBpfClient\n\t/workspace/pkg/ebpf/bpf_client.go:179\ngithub.com/aws/aws-network-policy-agent/controllers.NewPolicyEndpointsReconciler\n\t/workspace/controllers/policyendpoints_controller.go:93\nmain.main\n\t/workspace/main.go:93\nruntime.main\n\t/root/sdk/go1.20.4/src/runtime/proc.go:250"}
{"level":"error","timestamp":"2023-11-08T14:51:35.572Z","logger":"setup","msg":"unable to setup controller","controller":"PolicyEndpoints init failed","error":"failed to load maps","stacktrace":"main.main\n\t/workspace/main.go:97\nruntime.main\n\t/root/sdk/go1.20.4/src/runtime/proc.go:250"}

In the worker nodes under /var/log/aws-routed-eni/ebpf-sdk.log

{"level":"error","ts":"2023-11-08T14:51:35.572Z","caller":"elfparser/elf.go:144","msg":"Unable to create map and ret -1 and err permission denied"}
{"level":"error","ts":"2023-11-08T14:51:35.572Z","caller":"elfparser/elf.go:574","msg":"failed to create map unable to create map: permission denied"}

[TrevorH]

From wikipedia: "eBPF is a kernel technology (fully available since Linux 4.4)"

CentOS 7 has kernel 3.10.0 so it's just too old.

[monesh]

Thanks for the reply. Actually by default kernel was 3.10, but I have manually upgraded the kernel version to 6.5.10-1.el7.elrepo.x86_64 because NetworkPolicy supports only from kernel 5.10+ versions.

[jlehtone]

Have installed Kernel version 5.10+.
I have tried the same steps with Linux and Ubuntu os

How did you install that non-standard kernel (EDIT: I see, kernel-ml from ELRepo) and are you actually running it?
(I do agree with Trevor that you probably should not look at CentOS 7 any more since it dies before next summer.)

What is that "Linux"?

[monesh]

Yes, I have installed main line version of kernel-6.5.10-1.el7.elrepo.x86_64 from www.elrepo.org repository.

What is that "Linux"?

I have tried with AWS Linux 2 which is Red Hat Enterprise Linux (RHEL) 7, and on Ubuntu-20. In both it was working fine.
But my requirement is to get it done on CentOS7.

[jlehtone]

If AWS Linux 2 IS-A RHEL 7 and CentOS 7 is bug-for-bug compatible with RHEL 7, then there should be no issue.
if

[TrevorH]

Amazon Linux 2 is loosely based on RHEL 7 but includes many components from e.g. Fedora. It's not the same thing.

Also, what is the output from uname -r on this system?

[monesh]

Also, what is the output from uname -r on this system?

uname -r output is 6.5.10-1.el7.elrepo.x86_64

[monesh]

I have tried calico instead of AWS VPC CNI for Network Policy solution on this same CentOS image with old Kernel version, and it is working fine.
https://github.com/projectcalico/calico ... rts/calico

AWS VPC-CNI plugin is working fine on linux(RHEL 7) and ubuntu(v20) images, but not on CentOS.
Calico plugin is working fine on CentOS, but AWS VPC-CNI.

Which means, is there a compatible issue with CentOS and AWS VPC-CNI plugin?

[jlehtone]

Trevor did point out that WAS Linux 2 is not RHEL 7.

If genuine RHEL 7 image does work but CentOS 7 image does not, then that RHEL 7 image has something that CentOS 7 image does not.

Some applications do also read files like /etc/redhat-release and behave differently depending on the content.
No idea whether AWS VPC CNI is one of those.