Open Stack

Hello there,

TLDR;
we're about to switch to nftables[1], there are some changes described 
in the up-to-date doc. Reason is: faster, more modern interface, close 
to no possibility to get locked out of the system. Also, once it's 
switched, please use `nft list ruleset' instead of `iptables -L'.

[1] https://review.opendev.org/c/openstack/tripleo-heat-templates/+/852808

Longer version:

I've been working a good amount of time in order to get rid of the 
current tripleo_iptables custom action.

On of the reasons here was the fact we could get locked out the 
overcloud if, for any reason, a network reset happens in ansible during 
the application of the rules.

For the records, tripleo_iptables is calling the native "iptables" 
module from ansible, just doing a batching of the ruleset in an attempt 
to apply everything faster.
It has some weird things, such as reversing the rule order before 
applying them using the "insert" action, meaning it will lock everything 
first, then open one by one the accesses. Knowing "ssh" is like 003 
rule, you can imagine how things can go wrong.

So, I took the opportunity to make some changes. While doing so, I saw 
`iptables' was just a compatibility wrapper for nftables - basically, 
`iptables' is a symlink to `iptables-nft', so I also took the 
opportunity to bypass it, and go straight for `nft'.

Doing so, I created a new tripleo_nftables role in tripleo-ansible 
project; it's "just" creating files based on templates, then validates 
the whole lot of files, and applies all the rules in one single 
transaction. Compared to tripleo_iptables, it's really more robust, 
while being faster, and less prone to lock out and other unwanted things.

The current state is:
we're close, really close to switch things up. We're missing 2 patches 
in the CI to make the infra properly supported, and then, there's "the" 
switch itself.

What will change:
almost nothing: the way we create rules in tripleo-heat-templates and 
the different parameters therein doesn't change at all; the thing that 
will really change is the way to list the rules: instead of calling 
`iptables -vnL' or the like, you'll need to call `nft list ruleset'.

Now, in order to make things easier, the doc is already up-to-date:
https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/features/security_hardening.html#firewall-management

As you will see, there will be some differences in the actual layout: 
all of the tripleo rules will be in dedicated chains, prefixed by 
TRIPLEO_ - for instance, TRIPLEO_INPUT, TRIPLEO_OUTPUT, TRIPLEO_FORWARD, 
and so on. This allows to get a cleaner layout, cleaner way to filter 
the output and, really important thing, to ensure we're cleaning 
dangling rules - the TRIPLEO_* chains will be flushed before the ruleset 
are added. All in one single transaction.

Some more information/content:
tripleo_nftables role:
https://opendev.org/openstack/tripleo-ansible/src/branch/master/tripleo_ansible/roles/tripleo_nftables

Doc:
https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/features/security_hardening.html#firewall-management

Debug files available in the CI:
log of dropped packets:
https://logserver.rdoproject.org/54/31954/87/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-master/f14d78e/logs/undercloud/var/log/extra/dropped-packets.txt.gz

nftables configuration dump:
https://logserver.rdoproject.org/54/31954/87/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-master/f14d78e/logs/undercloud/var/log/extra/nftables.txt.gz

nftables configuration:
https://logserver.rdoproject.org/54/31954/87/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-master/f14d78e/logs/undercloud/etc/nftables/

All of that can help understanding the potential issues you may encounter.

As a side note, we've been testing the whole thing for about a month 
now, correcting issues, updating the doc and making sure at least all of 
the current CI jobs (yes: ALL) are green without any weird behavior.
A doc has been created in order to list the current state, with what 
we've seen, what we've done:
https://hackmd.io/F0W2gYw_SiaiWkowjFU9cw?view#NFTABLES-testing-results

Brace yourself, the change is coming :). But it should be transparent ;).



-- 
Cédric Jeanneret (He/Him/His)
Sr. Software Engineer - OpenStack Platform
Deployment Framework TC
Red Hat EMEA
https://www.redhat.com/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 840 bytes
Desc: OpenPGP digital signature
URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20220913/98c0ca74/attachment.sig>