Dear folks, Recently, we met some neutron networking problems in our envrionment, openstack version is zed, and kolla-ansible as the deployment tool. 1. Neutron BGP agent doesn't advertise the floating IPs to the BGP peer, in case of the floating IPs were served for port forwarding, but the floating IPs attached to VM/Container were advertised correctly. so the question is this scenario supported by BGP agent, if not when will it be supported, is it in the plan? 2. iptable rules restoring error in l3-agent and openvswitch-agent (A bug was reported in launchpad: https://bugs.launchpad.net/neutron/+bug/2024976) Bug #2024976 “iptable rules restoring error in l3-agent and open...” : Bugs : neutron<https://bugs.launchpad.net/neutron/+bug/2024976> Openstack version: zed/stable OS version: Ubuntu 22.04.2 LTS Kernel version: 5.15.0-75-generic #82-Ubuntu Deployment: kolla-ansible iptable rules restoring error in l3-agent and openvswitch-agent: openvswitch-agnet log: 2023-06-23 15:54:58.616 7 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [None req-4440bce1-8c07-4243-ac1b-2566b406a30a - - - - - -] Error while processing VIF ports: neutron_lib.exceptions.ProcessExecutionError: Exit code: 2; Cmd: [... bugs.launchpad.net openvswitch-agnet log: 2023-06-23 15:54:58.616 7 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [None req-4440bce1-8c07-4243-ac1b-2566b406a30a - - - - - -] Error while processing VIF ports: neutron_lib.exceptions.ProcessExecutionError: Exit code: 2; Cmd: ['iptables-restore', '-n']; Stdin: # Generated by iptables_manager *filter :FORWARD - [0:0] :INPUT - [0:0] :OUTPUT - [0:0] :neutron-filter-top - [0:0] :neutron-openvswi-FORWARD - [0:0] :neutron-openvswi-INPUT - [0:0] :neutron-openvswi-OUTPUT - [0:0] :neutron-openvswi-local - [0:0] :neutron-openvswi-sg-chain - [0:0] :neutron-openvswi-sg-fallback - [0:0] -I FORWARD 1 -j neutron-filter-top -I FORWARD 2 -j neutron-openvswi-FORWARD -I INPUT 1 -j neutron-openvswi-INPUT -I OUTPUT 1 -j neutron-filter-top -I OUTPUT 2 -j neutron-openvswi-OUTPUT -I neutron-filter-top 1 -j neutron-openvswi-local -I neutron-openvswi-FORWARD 1 -m physdev --physdev-out tap2fcacaf9-9d --physdev-is-bridged -m comment --comment "Accept all packets when port is trusted." -j ACCEPT -I neutron-openvswi-FORWARD 2 -m physdev --physdev-in tap2fcacaf9-9d --physdev-is-bridged -m comment --comment "Accept all packets when port is trusted." -j ACCEPT -I neutron-openvswi-FORWARD 3 -m physdev --physdev-out tap8c64cce3-ea --physdev-is-bridged -m comment --comment "Accept all packets when port is trusted." -j ACCEPT -I neutron-openvswi-FORWARD 4 -m physdev --physdev-in tap8c64cce3-ea --physdev-is-bridged -m comment --comment "Accept all packets when port is trusted." -j ACCEPT -I neutron-openvswi-sg-chain 1 -j ACCEPT -I neutron-openvswi-sg-fallback 1 -m comment --comment "Default drop rule for unmatched traffic." -j DROP COMMIT # Completed by iptables_manager # Generated by iptables_manager *raw :OUTPUT - [0:0] :PREROUTING - [0:0] :neutron-openvswi-OUTPUT - [0:0] :neutron-openvswi-PREROUTING - [0:0] -I OUTPUT 1 -j neutron-openvswi-OUTPUT -I PREROUTING 1 -j neutron-openvswi-PREROUTING COMMIT # Completed by iptables_manager ; Stdout: ; Stderr: iptables-restore v1.8.7 (nf_tables): Couldn't load match `physdev':No such file or directory Error occurred at line: 19 Try `iptables-restore -h' or 'iptables-restore --help' for more information. l3-agent log: 2023-06-23 16:15:49.545 33 ERROR neutron.agent.linux.iptables_manager [-] Failure applying iptables rules: neutron_lib.exceptions.ProcessExecutionError: Exit code: 2; Cmd: ['ip', 'netns', 'exec', 'qrouter-0f0e60d0-bf51-4361-901b-4b998201b44b', 'iptables-restore', '-n']; Stdin: # Generated by iptables_manager *filter :FORWARD - [0:0] :INPUT - [0:0] :OUTPUT - [0:0] :neutron-filter-top - [0:0] :neutron-l3-agent-FORWARD - [0:0] :neutron-l3-agent-INPUT - [0:0] :neutron-l3-agent-OUTPUT - [0:0] :neutron-l3-agent-local - [0:0] :neutron-l3-agent-scope - [0:0] -I FORWARD 1 -j neutron-filter-top -I FORWARD 2 -j neutron-l3-agent-FORWARD -I INPUT 1 -j neutron-l3-agent-INPUT -I OUTPUT 1 -j neutron-filter-top -I OUTPUT 2 -j neutron-l3-agent-OUTPUT -I neutron-filter-top 1 -j neutron-l3-agent-local -I neutron-l3-agent-FORWARD 1 -j neutron-l3-agent-scope -I neutron-l3-agent-scope 1 -m mark --mark 0x1/0xffff -j DROP COMMIT # Completed by iptables_manager # Generated by iptables_manager *mangle :FORWARD - [0:0] :INPUT - [0:0] :OUTPUT - [0:0] :POSTROUTING - [0:0] :PREROUTING - [0:0] :neutron-l3-agent-FORWARD - [0:0] :neutron-l3-agent-INPUT - [0:0] :neutron-l3-agent-OUTPUT - [0:0] :neutron-l3-agent-POSTROUTING - [0:0] :neutron-l3-agent-PREROUTING - [0:0] :neutron-l3-agent-float-snat - [0:0] :neutron-l3-agent-floatingip - [0:0] :neutron-l3-agent-mark - [0:0] :neutron-l3-agent-scope - [0:0] -I FORWARD 1 -j neutron-l3-agent-FORWARD -I INPUT 1 -j neutron-l3-agent-INPUT -I OUTPUT 1 -j neutron-l3-agent-OUTPUT -I POSTROUTING 1 -j neutron-l3-agent-POSTROUTING -I PREROUTING 1 -j neutron-l3-agent-PREROUTING -I neutron-l3-agent-PREROUTING 1 -j neutron-l3-agent-mark -I neutron-l3-agent-PREROUTING 2 -j neutron-l3-agent-scope -I neutron-l3-agent-PREROUTING 3 -m connmark ! --mark 0x0/0xffff0000 -j CONNMARK --restore-mark --nfmask 0xffff0000 --ctmask 0xffff0000 -I neutron-l3-agent-PREROUTING 4 -j neutron-l3-agent-floatingip -I neutron-l3-agent-PREROUTING 5 -d 169.254.169.254/32 -i qr-+ -p tcp -m tcp --dport 80 -j MARK --set-xmark 0x1/0xffff -I neutron-l3-agent-float-snat 1 -m connmark --mark 0x0/0xffff0000 -j CONNMARK --save-mark --nfmask 0xffff0000 --ctmask 0xffff0000 COMMIT # Completed by iptables_manager # Generated by iptables_manager *nat :OUTPUT - [0:0] :POSTROUTING - [0:0] :PREROUTING - [0:0] :neutron-l3-agent-OUTPUT - [0:0] :neutron-l3-agent-POSTROUTING - [0:0] :neutron-l3-agent-PREROUTING - [0:0] :neutron-l3-agent-float-snat - [0:0] :neutron-l3-agent-snat - [0:0] :neutron-postrouting-bottom - [0:0] -I OUTPUT 1 -j neutron-l3-agent-OUTPUT -I POSTROUTING 1 -j neutron-l3-agent-POSTROUTING -I POSTROUTING 2 -j neutron-postrouting-bottom -I PREROUTING 1 -j neutron-l3-agent-PREROUTING -I neutron-l3-agent-POSTROUTING 1 ! -o rfp-0f0e60d0-b -m conntrack ! --ctstate DNAT -j ACCEPT -I neutron-l3-agent-PREROUTING 1 -d 137.175.31.207/32 -i rfp-0f0e60d0-b -j DNAT --to-destination 10.10.0.246 -I neutron-l3-agent-float-snat 1 -s 10.10.0.246/32 -j SNAT --to-source 137.175.31.207 --random-fully -I neutron-l3-agent-snat 1 -j neutron-l3-agent-float-snat -I neutron-postrouting-bottom 1 -m comment --comment "Perform source NAT on outgoing traffic." -j neutron-l3-agent-snat COMMIT # Completed by iptables_manager # Generated by iptables_manager *raw :OUTPUT - [0:0] :PREROUTING - [0:0] :neutron-l3-agent-OUTPUT - [0:0] :neutron-l3-agent-PREROUTING - [0:0] -I OUTPUT 1 -j neutron-l3-agent-OUTPUT -I PREROUTING 1 -j neutron-l3-agent-PREROUTING COMMIT # Completed by iptables_manager ; Stdout: ; Stderr: iptables-restore v1.8.7 (nf_tables): Couldn't load match `mark':No such file or directory Error occurred at line: 19 And we check the system the x_tables kernel module were loaded: # lsmod | grep x_tables x_tables 53248 12 xt_conntrack,nft_compat,xt_tcpudp,xt_physdev,xt_nat,xt_comment,ip6_tables,xt_connmark,xt_CT,ip_tables,xt_REDIRECT,xt_mark (neutron-l3-agent)[neutron@compute06 usr]$ find . -name "*mark.so" ./lib/x86_64-linux-gnu/xtables/libxt_connmark.so ./lib/x86_64-linux-gnu/xtables/libxt_mark.so ./lib/x86_64-linux-gnu/xtables/libebt_mark.so (neutron-l3-agent)[neutron@compute06 usr]$ find . -name "*physdev.so" ./lib/x86_64-linux-gnu/xtables/libxt_physdev.so Does someone have ever met the problems what is the solution the resovle them. Thanks in advance Sincerely， Bryan