firewall-cmd - add-forward-port don't work

[eduardolucioac]

Some relly useful documentation from libvirt...
https://wiki.libvirt.org/page/Networkin ... onnections
https://libvirt.org/firewall.html

[jlehtone]

CentOS 7 has libvirt-4.5.0 and firewalld-0.5.3. Most of the lovely features mentioned in the second document are not available.

The "hook hack" from the wiki seems necessary. Note that libvirt executes a script and the script can call firewall-cmd (--direct --passthrough), rather than iptables as in the example.

What you want is that when you boot host and guests start, the rules will be correct; without manual intervention.
Furthermore, stopping and (re)starting guests should maintain that too.

[eduardolucioac]

THE REAL PROBLEM:

The command...

Code: Select all

firewall-cmd --permanent --zone=public --direct --add-rule ipv4 filter FORWARD 0 -d 0.0.0.0/0 -j ACCEPT
firewall-cmd --permanent --zone=public --add-forward-port=port=222:proto=tcp:toport=22:toaddr=10.1.0.9
firewall-cmd --reload

... works just as well as firewalld (used by CentOS 7).

The problem is in the KVM (libvirt) which when starting the "network" virbr0 injects rules (probably via firewalld) which prevents (first match principle) any attempt to port forwarding by conventional ways using firewall-cmd. The KVM injected (libvirt) rules are at the top of FORWARD and POSTROUTING, before regular firewalld ruleset. Since the netfilter operates on first match principle, these rules.

SOLUTION:

Due to the problems generated by the rules injected by KVM (libvirt) into the firewall the solution is use a hack/workaround called libvirt-hook-qemu as we explained below...

Download and install libvirt-hook-qemu...

Code: Select all

yum -y install git-core
cd /usr/local/src
git clone https://github.com/saschpe/libvirt-hook-qemu.git
cd libvirt-hook-qemu
make install
rm -rf libvirt-hook-qemu

List your virtual machines (guests)...

Code: Select all

virsh list --all

... and copy the name of the target virtual machine.

Turn off the virtual machine...

Code: Select all

virsh shutdown "<VM_NAME>" --mode acpi

Open the file "hooks.json", delete its contents...

Code: Select all

vi /etc/libvirt/hooks/hooks.json

... and enter your rules as the template below (other ways of setting up are possible)...

Code: Select all

{
    "<VM_NAME>": {
        "interface": "virbr0",
        "private_ip": "<VM_IP>",
        "port_map": {
            "tcp": [[<PORT_ON_HOST>, PORT_ON_GUEST>]]
        }
    }
}

Restart libvirtd...

Code: Select all

systemctl restart libvirtd.service

... and start the virtual machine...

Code: Select all

virsh start "<VM_NAME>"

Done! =D

------------

FURTHER:

This KVM (libvirt) behavior - injects rules which prevents any attempt to port forwarding - is IMPOSITIVE, INVASIVE AND UNNECESSARY... First because since there is no port forwarding on the host no virtual machine will be reachable on the default network (virbr0) and second because it forces us to use a hack/workaround (or "gato"/"gambiarra" as we call it in Brazil =D ).

In our view makes no sense impose the user how he should build his infrastructure or not... This makes no sense! To put it bluntly, we think it would be best if these settings were optional (injects rules...) as other leading hypervisors do. And we go further... If is to KVM (libvirt) controlling this kind of thing then it should also offer ways to map ports to virtual machines..

The KVM (libvirt) is a GREAT tool, but not all users are willing - for various reasons - to place their machines directly on the host machine's network just as not all users are willing to give up all the flexibility/accessibility that the default network (virbr0) offers.

Thanks!

[Refs.: https://bugzilla.redhat.com/show_bug.cgi?id=846810 ,
https://libvirt.org/firewall.html ,
https://libvirt.org/formatnetwork.html#examplesRoute,
https://saschpe.wordpress.com/2014/03/0 ... -networks/ ,
https://serverfault.com/q/170079/276753 ,
https://serverfault.com/q/915257/276753 ,
https://wiki.libvirt.org/page/VirtualNe ... outed_mode ,
viewtopic.php?f=50&t=71454&sid=7f5190a2 ... 3dc404a47f ,
https://www.cyberciti.biz/faq/kvm-forwa ... -on-linux/ ]

[jlehtone]

To be honest, we cannot blame libvirt, for the current upstream has resolved the issue. We could blame libvirt as it was before 2014 for being too simple back then. That simple has been enough for most users. There are other virtualization platforms with more elaborate control.

We cannot blame Red Hat either for keeping its promise that they won't introduce major feature changes during lifetime of a major version.


There is philosophical question about:

Code: Select all

-I FORWARD 0 -j ACCEPT

That did allow routing of everything by the host. No filtering.

There is still the NAT for obfuscation, but in principle an attacker can have static route:

Code: Select all

10.1.0.0/24 via 172.16.13.8

to reach every port of every guest.

The question is: should the host filter anything at all? Simply route everything and let each guest firewall on their own.
An another extreme is that the guests trust host to firewall everything and guest firewalls are disabled.

[eduardolucioac]

I am a free software enthusiast and support initiatives like KVM (libvirt), just note the care I took to post a clear solution - with your help of course - to the issue at hand! I'm not blaming anyone because it doesn't matter... Paraphrasing you is a "philosophical question"... I don't see as a good thing something that makes the user waste so much time. It is a constructive criticism that I make... =|

[...] the current upstream has resolved the issue. [...]

Excellent news! I am glad you are careful to record this!

By the way ... Thank you so much for all your help @jlehtone!!!! =D
See: https://superuser.com/a/1475915/195840