May 2022 - openstack-discuss - lists.openstack.org

Hi, I am very happy to meet all of you. I hope to exchange more knowledge about openstack with you.
by Tom NewChao 10 May '22

10 May '22

I am very happy to meet all of you. I hope to exchange more knowledge about openstack with you.

1 0

Hi, Nice to meet you, I am very happly to find faimly.
by 朱超 10 May '22

10 May '22

Hi, Nice to meet you, I am very happly to find faimly.

1 0

Neutron + OVN raft cluster
by Tiago Pires 10 May '22

10 May '22

Hi all, I was checking the mail list history and this thread https://mail.openvswitch.org/pipermail/ovs-discuss/2018-March/046438.html caught my attention about raft ovsdb clustering. In my setup (OVN 20.03 and Openstack Ussuri) on the ovn-controller we have configured the ovn-remote="tcp:10.2X.4X.4:6642,tcp:10.2X.4X.68:6642,tcp:10.2X.4X.132:6642" with the 3 OVN central member that they are in cluster mode. Also on the neutron ML2 side: [ovn] ovn_native_dhcp = True ovn_nb_connection = tcp:10.2X.4X.4:6641,tcp:10.2X.4X.68:6641,tcp:10.2X.4X.132:6641 ovn_sb_connection = tcp:10.2X.4X.4:6642,tcp:10.2X.4X.68:6642,tcp:10.2X.4X.132:6642 We are experiencing an issue with Neutron when the OVN leader decide to take a snapshot and by design another member became leader(more less every 8 minutes): 2022-05-05T16:57:42.135Z|17401|raft|INFO|Transferring leadership to write a snapshot. ovs-appctl -t /var/run/ovn/ovnsb_db.ctl cluster/status OVN_Southbound 4a03 Name: OVN_Southbound Cluster ID: ca74 (ca744caf-40cd-4751-a2f2-86e35ad6541c) Server ID: 4a03 (4a0328dc-e9a4-495e-a4f1-0a0340fc6d19) Address: tcp:10.2X.4X.132:6644 Status: cluster member Role: leader Term: 1912 Leader: self Vote: self Election timer: 10000 Log: [497643, 498261] Entries not yet committed: 0 Entries not yet applied: 0 Connections: ->3d6c ->4ef0 <-3d6c <-4ef0 Servers: 4a03 (4a03 at tcp:10.2X.4X.132:6644) (self) next_index=497874 match_index=498260 3d6c (3d6c at tcp:10.2X.4X.68:6644) next_index=498261 match_index=498260 4ef0 (4ef0 at tcp:10.2X.4X.4:6644) next_index=498261 match_index=498260 As I understood the tcp connections from the Neutron (NB) and ovn-controllers (SB) to OVN Central are established only with the leader: #OVN central leader $ netstat -nap | grep 6642| more tcp 0 0 0.0.0.0:6642 0.0.0.0:* LISTEN - tcp 0 0 10.2X.4X.132:6642 10.24.40.17:47278 ESTABLISHED - tcp 0 0 10.2X.4X.132:6642 10.24.40.76:36240 ESTABLISHED - tcp 0 0 10.2X.4X.132:6642 10.2X.4X.17:47280 ESTABLISHED - tcp 0 0 10.2X.4X.132:6642 10.2X.4X.6:43102 ESTABLISHED - tcp 0 0 10.2X.4X.132:6642 10.2X.4X.75:58890 ESTABLISHED - tcp 0 0 10.2X.4X.132:6642 10.2X.4X.6:43108 ESTABLISHED - tcp 0 0 10.2X.4X.132:6642 10.2X.4X.17:47142 ESTABLISHED - tcp 0 0 10.2X.4X.132:6642 10.2X.4X.71:48808 ESTABLISHED - tcp 0 0 10.2X.4X.132:6642 10.2X.4X.17:47096 ESTABLISHED - #OVN follower 2 $ netstat -nap | grep 6642 tcp 0 0 0.0.0.0:6642 0.0.0.0:* LISTEN - tcp 0 0 10.2X.4X.4:6642 10.2X.4X.76:57256 ESTABLISHED - tcp 0 0 10.2X.4X.4:6642 10.2X.4X.134:54026 ESTABLISHED - tcp 0 0 10.2X.4X.4:6642 10.2X.4X.10:34962 ESTABLISHED - tcp 0 0 10.2X.4X.4:6642 10.2X.4X.6:49238 ESTABLISHED - tcp 0 0 10.2X.4X.4:6642 10.2X.4X.135:59972 ESTABLISHED - tcp 0 0 10.2X.4X.4:6642 10.2X.4X.75:40162 ESTABLISHED - tcp 0 0 10.2X.4X.4:39566 10.2X.4X.132:6642 ESTABLISHED - #OVN follower 3 netstat -nap | grep 6642 tcp 0 0 0.0.0.0:6642 0.0.0.0:* LISTEN - tcp 0 0 10.2X.4X.68:6642 10.2X.4X.70:40750 ESTABLISHED - tcp 0 0 10.2X.4X.68:6642 10.2X.4X.11:49718 ESTABLISHED - tcp 0 0 10.2X.4X.68:45632 10.2X.4X.132:6642 ESTABLISHED - tcp 0 0 10.2X.4X.68:6642 10.2X.4X.16:44816 ESTABLISHED - tcp 0 0 10.2X.4X.68:6642 10.2X.4X.7:45216 ESTABLISHED The issue that we are experiencing is on the neutron-server that disconnects when there is the ovn leader change (due snapshot like each 8 minutes) and reconnects to the next leader. It breaks the Openstack API when someone is trying to create a VM at the same time. First, is my current configuration correct? Should the leader change and break the neutron side? Or is there some missing configuration? I was wondering if it is possible to use a LB with VIP and this VIP balance the connections to the ovn central members and I would reconfigure on the neutron side only with the VIP and also on the ovs-controllers. Does that make sense? Thank you. Regards, Tiago Pires

4 5

[venus] Yoga tarball is still missing
by Radosław Piliszek 09 May '22

09 May '22

Dear Venus team, The Yoga tarball is still missing. [1] As we are releasing Kolla now, we are forced to temporarily exclude Venus from the Yoga release. [1] https://tarballs.opendev.org/openstack/venus/ Kind regards, -yoctozepto

3 2

[skyline] SSO support
by Martin Chlumsky 09 May '22

09 May '22

Hello, We are evaluating Skyline and one major blocker is Single sign-on (SSO) support (we currently federate Keystone/Horizon with AzureAD). I searched through the git repositories and couldn't find any relevant mention of SSO or federation (openidc or saml). Are there plans to support this feature (or is it supported and I just missed it somehow)? Thank you, Martin

2 2

[TripleO] Tear down of Train centOS 7 check jobs and integration line
by Ronelle Landy 09 May '22

09 May '22

Hello All, We have been running check/gate testing and integration lines for the Train release on both CentOS7 and CentOS 8. Following the work on CentOS 9 and the longevity of the Train release, we proposed removing the CentOS 7 Train jobs and building/testing changes to this release on CentOS 8 only. We floated this proposal with some interested parties and received no objections so we are beginning work on this tear down. We anticipate that removing the CentOS 7 jobs will allow patches to merge quicker and will free up resources for future work. Please respond if you have any questions or concerns. Thanks, Ronelle (for the TripleO CI team)

2 2

[neutron] Bug Deputy Report May 03 - 09
by Elvira Garcia Ruiz 09 May '22

09 May '22

Hi, I was the bug deputy last week. Find the summary below: High: ------- - [OVN] Add support for Baremetal provisioning with ML2/OVN https://bugs.launchpad.net/neutron/+bug/1971431 Proposed fix: https://review.opendev.org/c/openstack/neutron/+/840287 - [neutron][api] test_log_deleted_with_corresponding_security_group failing randomly https://bugs.launchpad.net/neutron/+bug/1971569 Unassigned - [FT] Error in "test_virtual_port_host_update" https://bugs.launchpad.net/neutron/+bug/1971672 Unassigned Medium: ----------- - [api] add port_forwarding_id when list floatingip https://bugs.launchpad.net/neutron/+bug/1971646 Fix proposed: https://review.opendev.org/c/openstack/neutron/+/840565 RFE: ------- - [RFE][fwaas][OVN]support l3 firewall for ovn driver https://bugs.launchpad.net/neutron/+bug/1971958 Invalid: --------- - [Neutron][OpenContrail]Trunk ports don't have standard attributes. https://bugs.launchpad.net/neutron/+bug/1971697 Looks like a request for help with an external plugin, I redirected them to the openstack discuss mailing list since I have no idea of how the OpenContrail driver works. Undecided: --------------- - evacuation failure causes POST_FAILURE in nova-ovs-hybrid-plug job https://bugs.launchpad.net/nova/+bug/1971563 I think we might not need to do anything else regarding this bug from the Neutron side, I asked Sean to be sure just in case. Kind regards! Elvira

1 0

[ops][octavia][neutron] Distributed Virtual Routing: Floating IPs attached to virtual IP addresses are assigned on network nodes
by Jan Horstmann 09 May '22

09 May '22

Hello! When we initially deployed openstack we thought that using distributed virtual routing with ml2/ovs-dvr would give us the ability to automatically scale our network capacity with the number of hypervisors we use. Our main workload are kubernetes clusters which receive ingress traffic via octavia loadbancers (configured to use the amphora driver). So the idea was that we could increase the number of loadbalancers to spread the traffic over more and more compute nodes. This would imply that any volume based (distributed) denial of service attack on a single loadbalancer would just saturate a single compute node and leave the rest of the system functional. We have recently learned that, no matter the loadbalancer topology, a virtual IP is created for it by octavia. This, and probably all virtual IPs in openstack, are reserved by an unbound and disabled port and then set as an allowed address pair on any server's port which might hold it. Up to this point our initial assumption should be true, as the server actually holding the virtual IP would reply to any ARP requests and thus any traffic should be routed to the node with the virtual machine of the octavia amphora. However, we are using our main provider network as a floating IP pool and do not allow direct port creation. When a floating IP is attached to the virtual IP it is assigned to the SNAT router namespace on a network node. Naturally in high traffic or even (distributed) denial of service situations the network node might become a bottleneck. A situation we thought we could avoid by using distributed virtual routing in the first place. This leads me to a rabbit hole of questions I hope someone might be able to help with: Is the assessment above correct or am I missing something? If it is correct, do we have any other options than vertically scaling our network nodes to handle traffic? Do other ml2 drivers (e.g. OVN) handle this scenario differently? If our network nodes need to handle most of the traffic anyway, do we still have any advantage using distributed virtual routing? Especially when considering the increased complexity compared to a non- distributed setup? Has anyone ever explored non-virtual IP based high availability options like e.g. BGP multipathing in a distributed virtual routing scenario? Any input is highly appreciated. Regards, Jan

3 3

Glance browser error
by Derek O keeffe 09 May '22

09 May '22

Hi all, Second mail about this issue. I'm getting the following error in the chrome console: Refused to connect to 'https://10.xx.xx.xxx:9292/v2/images/f7df36f3-33d8-49b6-95b6-b16e95bc3225/fi…' because it violates the following Content Security Policy directive: "default-src 'self'". Note that 'connect-src' was not explicitly set, so 'default-src' is used as a fallback I can't upload an image through horizon but I can do it from the cli with no issues. I can't find anything online to help with the issue so any input would be greatly appreciated. Regards,Derek

3 2

[magnum] Proposing Mohammed Naser for core-reviewer
by Spyros Trigazis 09 May '22

09 May '22

Dear all, I would like to nominate Mohammed Naser for core reviewer in the magnum project. Mohammed has a proven knowledge of the codebase with frequent contributions and reviews. Mohammed will be of great help in fast and reliable reviews. Cheers, Spyros

1 0