Open Stack

Dear folks,

Recently, we met some neutron networking problems in our envrionment, openstack version is zed, and kolla-ansible as the deployment tool.


  1.  Neutron BGP agent doesn't advertise the floating IPs to the BGP peer, in case of the floating IPs were served for port forwarding, but the floating IPs attached to VM/Container  were advertised correctly. so the question is this scenario supported by BGP agent, if not when will it be supported, is it in the plan?
  2.  iptable rules restoring error in l3-agent and openvswitch-agent (A bug was reported in launchpad: https://bugs.launchpad.net/neutron/+bug/2024976)
Bug #2024976 “iptable rules restoring error in l3-agent and open...” : Bugs : neutron<https://bugs.launchpad.net/neutron/+bug/2024976>
Openstack version: zed/stable OS version: Ubuntu 22.04.2 LTS Kernel version: 5.15.0-75-generic #82-Ubuntu Deployment: kolla-ansible iptable rules restoring error in l3-agent and openvswitch-agent:       openvswitch-agnet log: 2023-06-23 15:54:58.616 7 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [None req-4440bce1-8c07-4243-ac1b-2566b406a30a - - - - - -] Error while processing VIF ports: neutron_lib.exceptions.ProcessExecutionError: Exit code: 2; Cmd: [...
bugs.launchpad.net


      openvswitch-agnet log:

2023-06-23 15:54:58.616 7 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [None req-4440bce1-8c07-4243-ac1b-2566b406a30a - - - - - -] Error while processing VIF ports: neutron_lib.exceptions.ProcessExecutionError: Exit code: 2; Cmd: ['iptables-restore', '-n']; Stdin: # Generated by iptables_manager
*filter
:FORWARD - [0:0]
:INPUT - [0:0]
:OUTPUT - [0:0]
:neutron-filter-top - [0:0]
:neutron-openvswi-FORWARD - [0:0]
:neutron-openvswi-INPUT - [0:0]
:neutron-openvswi-OUTPUT - [0:0]
:neutron-openvswi-local - [0:0]
:neutron-openvswi-sg-chain - [0:0]
:neutron-openvswi-sg-fallback - [0:0]
-I FORWARD 1 -j neutron-filter-top
-I FORWARD 2 -j neutron-openvswi-FORWARD
-I INPUT 1 -j neutron-openvswi-INPUT
-I OUTPUT 1 -j neutron-filter-top
-I OUTPUT 2 -j neutron-openvswi-OUTPUT
-I neutron-filter-top 1 -j neutron-openvswi-local
-I neutron-openvswi-FORWARD 1 -m physdev --physdev-out tap2fcacaf9-9d --physdev-is-bridged -m comment --comment "Accept all packets when port is trusted." -j ACCEPT
-I neutron-openvswi-FORWARD 2 -m physdev --physdev-in tap2fcacaf9-9d --physdev-is-bridged -m comment --comment "Accept all packets when port is trusted." -j ACCEPT
-I neutron-openvswi-FORWARD 3 -m physdev --physdev-out tap8c64cce3-ea --physdev-is-bridged -m comment --comment "Accept all packets when port is trusted." -j ACCEPT
-I neutron-openvswi-FORWARD 4 -m physdev --physdev-in tap8c64cce3-ea --physdev-is-bridged -m comment --comment "Accept all packets when port is trusted." -j ACCEPT
-I neutron-openvswi-sg-chain 1 -j ACCEPT
-I neutron-openvswi-sg-fallback 1 -m comment --comment "Default drop rule for unmatched traffic." -j DROP
COMMIT
# Completed by iptables_manager
# Generated by iptables_manager
*raw
:OUTPUT - [0:0]
:PREROUTING - [0:0]
:neutron-openvswi-OUTPUT - [0:0]
:neutron-openvswi-PREROUTING - [0:0]
-I OUTPUT 1 -j neutron-openvswi-OUTPUT
-I PREROUTING 1 -j neutron-openvswi-PREROUTING
COMMIT
# Completed by iptables_manager
; Stdout: ; Stderr: iptables-restore v1.8.7 (nf_tables): Couldn't load match `physdev':No such file or directory

Error occurred at line: 19
Try `iptables-restore -h' or 'iptables-restore --help' for more information.

      l3-agent log:

2023-06-23 16:15:49.545 33 ERROR neutron.agent.linux.iptables_manager [-] Failure applying iptables rules: neutron_lib.exceptions.ProcessExecutionError: Exit code: 2; Cmd: ['ip', 'netns', 'exec', 'qrouter-0f0e60d0-bf51-4361-901b-4b998201b44b', 'iptables-restore', '-n']; Stdin: # Generated by iptables_manager
*filter
:FORWARD - [0:0]
:INPUT - [0:0]
:OUTPUT - [0:0]
:neutron-filter-top - [0:0]
:neutron-l3-agent-FORWARD - [0:0]
:neutron-l3-agent-INPUT - [0:0]
:neutron-l3-agent-OUTPUT - [0:0]
:neutron-l3-agent-local - [0:0]
:neutron-l3-agent-scope - [0:0]
-I FORWARD 1 -j neutron-filter-top
-I FORWARD 2 -j neutron-l3-agent-FORWARD
-I INPUT 1 -j neutron-l3-agent-INPUT
-I OUTPUT 1 -j neutron-filter-top
-I OUTPUT 2 -j neutron-l3-agent-OUTPUT
-I neutron-filter-top 1 -j neutron-l3-agent-local
-I neutron-l3-agent-FORWARD 1 -j neutron-l3-agent-scope
-I neutron-l3-agent-scope 1 -m mark --mark 0x1/0xffff -j DROP
COMMIT
# Completed by iptables_manager
# Generated by iptables_manager
*mangle
:FORWARD - [0:0]
:INPUT - [0:0]
:OUTPUT - [0:0]
:POSTROUTING - [0:0]
:PREROUTING - [0:0]
:neutron-l3-agent-FORWARD - [0:0]
:neutron-l3-agent-INPUT - [0:0]
:neutron-l3-agent-OUTPUT - [0:0]
:neutron-l3-agent-POSTROUTING - [0:0]
:neutron-l3-agent-PREROUTING - [0:0]
:neutron-l3-agent-float-snat - [0:0]
:neutron-l3-agent-floatingip - [0:0]
:neutron-l3-agent-mark - [0:0]
:neutron-l3-agent-scope - [0:0]
-I FORWARD 1 -j neutron-l3-agent-FORWARD
-I INPUT 1 -j neutron-l3-agent-INPUT
-I OUTPUT 1 -j neutron-l3-agent-OUTPUT
-I POSTROUTING 1 -j neutron-l3-agent-POSTROUTING
-I PREROUTING 1 -j neutron-l3-agent-PREROUTING
-I neutron-l3-agent-PREROUTING 1 -j neutron-l3-agent-mark
-I neutron-l3-agent-PREROUTING 2 -j neutron-l3-agent-scope
-I neutron-l3-agent-PREROUTING 3 -m connmark ! --mark 0x0/0xffff0000 -j CONNMARK --restore-mark --nfmask 0xffff0000 --ctmask 0xffff0000
-I neutron-l3-agent-PREROUTING 4 -j neutron-l3-agent-floatingip
-I neutron-l3-agent-PREROUTING 5 -d 169.254.169.254/32 -i qr-+ -p tcp -m tcp --dport 80 -j MARK --set-xmark 0x1/0xffff
-I neutron-l3-agent-float-snat 1 -m connmark --mark 0x0/0xffff0000 -j CONNMARK --save-mark --nfmask 0xffff0000 --ctmask 0xffff0000
COMMIT
# Completed by iptables_manager
# Generated by iptables_manager
*nat
:OUTPUT - [0:0]
:POSTROUTING - [0:0]
:PREROUTING - [0:0]
:neutron-l3-agent-OUTPUT - [0:0]
:neutron-l3-agent-POSTROUTING - [0:0]
:neutron-l3-agent-PREROUTING - [0:0]
:neutron-l3-agent-float-snat - [0:0]
:neutron-l3-agent-snat - [0:0]
:neutron-postrouting-bottom - [0:0]
-I OUTPUT 1 -j neutron-l3-agent-OUTPUT
-I POSTROUTING 1 -j neutron-l3-agent-POSTROUTING
-I POSTROUTING 2 -j neutron-postrouting-bottom
-I PREROUTING 1 -j neutron-l3-agent-PREROUTING
-I neutron-l3-agent-POSTROUTING 1 ! -o rfp-0f0e60d0-b -m conntrack ! --ctstate DNAT -j ACCEPT
-I neutron-l3-agent-PREROUTING 1 -d 137.175.31.207/32 -i rfp-0f0e60d0-b -j DNAT --to-destination 10.10.0.246
-I neutron-l3-agent-float-snat 1 -s 10.10.0.246/32 -j SNAT --to-source 137.175.31.207 --random-fully
-I neutron-l3-agent-snat 1 -j neutron-l3-agent-float-snat
-I neutron-postrouting-bottom 1 -m comment --comment "Perform source NAT on outgoing traffic." -j neutron-l3-agent-snat
COMMIT
# Completed by iptables_manager
# Generated by iptables_manager
*raw
:OUTPUT - [0:0]
:PREROUTING - [0:0]
:neutron-l3-agent-OUTPUT - [0:0]
:neutron-l3-agent-PREROUTING - [0:0]
-I OUTPUT 1 -j neutron-l3-agent-OUTPUT
-I PREROUTING 1 -j neutron-l3-agent-PREROUTING
COMMIT
# Completed by iptables_manager
; Stdout: ; Stderr: iptables-restore v1.8.7 (nf_tables): Couldn't load match `mark':No such file or directory

Error occurred at line: 19

      And we check the system the x_tables kernel module were loaded:

# lsmod | grep x_tables
x_tables               53248  12 xt_conntrack,nft_compat,xt_tcpudp,xt_physdev,xt_nat,xt_comment,ip6_tables,xt_connmark,xt_CT,ip_tables,xt_REDIRECT,xt_mark

(neutron-l3-agent)[neutron at compute06 usr]$ find . -name "*mark.so"
./lib/x86_64-linux-gnu/xtables/libxt_connmark.so
./lib/x86_64-linux-gnu/xtables/libxt_mark.so
./lib/x86_64-linux-gnu/xtables/libebt_mark.so
(neutron-l3-agent)[neutron at compute06 usr]$ find . -name "*physdev.so"
./lib/x86_64-linux-gnu/xtables/libxt_physdev.so

Does someone have ever met the problems what is the solution the resovle them. Thanks in advance



Sincerely，



Bryan

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20230625/489ed01f/attachment-0001.htm>