December 2022 - openstack-discuss

[qa] Cancelling next office hours
by Martin Kopec 16 Dec '22

16 Dec '22

Hi everyone, we're cancelling office hours for the next 3 weeks due to the holidays. The next office hour is gonna be on January 10th. Thank you. Enjoy your holidays! -- Martin Kopec Senior Software Quality Engineer Red Hat EMEA IM: kopecmartin

1 0

kolla-ansible deploy glance with file as a backend on multiple nodes
by A Monster 15 Dec '22

15 Dec '22

I have made a deployment of openstack on multiple centos stream 8 nodes, and used files as a backend for glance service, but according to the documentation, when using file as a glance backend, It can only be deployed on a single node, what can I do to deploy it on multiple servers at once. Thank you.

3 2

reconfiguring the external interface to multiple interfaces doesn't work
by pradeep 15 Dec '22

15 Dec '22

Hello All, I have deployed OpenStack Yoga using kolla ansible with a single external interface initially. Later we realized that we wanted to use multiple interfaces. I have followed the docs to modify the external interface in the globals.yml file as below. *Before:* *globals.yml* neutron_external_interface: "bond2" *After:* *globals.yml* neutron_external_interface: "bond2,enp6s0f1" neutron_bridge_name: "br-ex,br-ex2" *Command used:* kolla-ansible -i multinode reconfigure --tags "common,horizon,neutron" -vv br-ex2 doesn't show up in the neutron nodes. I also noticed that my neutron-openvswitch-agent is continuously restating in both the neutron nodes. neutron-openvswitch-agent.log says as below 2022-12-15 02:12:35.121 7 INFO neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.ovs_bridge [-] Bridge br-int has datapath-ID 0000be5c93241c4f 2022-12-15 02:12:36.040 7 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [-] Mapping physical network physnet1 to bridge br-ex 2022-12-15 02:12:36.041 7 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [-] Bridge br-ex datapath-id = 0x0000801844eaf971 2022-12-15 02:12:36.057 7 INFO neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.ovs_bridge [-] Bridge br-ex has datapath-ID 0000801844eaf971 2022-12-15 02:12:36.063 7 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [-] Mapping physical network physnet2 to bridge br-ex2 2022-12-15 02:12:36.063 7 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [-] Bridge br-ex2 for physical network physnet2 does not exist. Agent terminated! Here is the config file. cat /etc/kolla/neutron-openvswitch-agent/openvswitch_agent.ini [agent] tunnel_types = vxlan l2_population = true arp_responder = true extensions = qos,sfc [securitygroup] firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver [ovs] bridge_mappings = physnet1:br-ex,physnet2:br-ex2 datapath_type = system ovsdb_connection = tcp:127.0.0.1:6640 ovsdb_timeout = 10 local_ip = <IP_Address> Also, i tried to change the bridge name from br-ex to br-ex1 but it still shows up br-ex (this can be perceived as my second issue). How can I resolve both issues? Kindly help.

1 0

stackalytics is stop update,
by 韩光宇 15 Dec '22

15 Dec '22

Hi, all I see stackalytics[1] is stop update. The page show "The data was last updated on 09 Nov 2022 12:44:22 UTC". Does anyone know why? And If you want to check the contribution statistics, is there still a way? Thank you for any help. Cheers, Han [1]: www.stackalytics.io

5 6

[manila][release] Proposing to EOL Stein
by Carlos Silva 15 Dec '22

15 Dec '22

Hello! Recently in a weekly meeting we chatted about EOLing stable/stein and all the attendants were found to be in favor of this [0]. As the procedures go [1], we need to make it formal through a post in this mailing list, and see if there are objections. In case of concerns or objections, please reach out through email or the #openstack-manila IRC channel. This will impact all branched manila repositories (manila, python-manilaclient and manila-ui). If there aren't objections or strong concerns, I will be proposing the patches to EOL stable/stein within one week. [0] https://meetings.opendev.org/meetings/manila/2022/manila.2022-11-03-15.00.l… [1] https://docs.openstack.org/project-team-guide/stable-branches.html#end-of-l… Thanks, carloss

1 1

[cinder] Festival of XS reviews?
by Rajat Dhasmana 14 Dec '22

14 Dec '22

Hello Argonauts, I've come to know informally that people might not be around during this time due to the holiday season. Based on this, I would like to ask the cinder team if we would like to have the festival of XS reviews this friday or not? Date: 16 December, 2022 Time: 1400-1600 UTC Please reply with your feedback and if we've enough people joining, we can continue with the session. Thanks Rajat Dhasmana

2 1

[cinder] Bug Report from 12-14-2022
by Sofia Enriquez 14 Dec '22

14 Dec '22

This is a bug report from 12-07-2022 to 12-14-2022. Agenda: https://etherpad.opendev.org/p/cinder-bug-squad-meeting ----------------------------------------------------------------------------------------- High - https://bugs.launchpad.net/cinder/+bug/1999252 "[RBD -> SolidFire] Cache mode was not updated after volume retype." Unassigned. Medium - https://bugs.launchpad.net/nova/+bug/1999125 "cinder retype volume raise exception with shutoff vm." Unassigned. Cheers, Sofia -- Sofía Enriquez she/her Software Engineer Red Hat PnT <https://www.redhat.com> IRC: @enriquetaso @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://www.redhat.com>

1 0

[neutron] Neutron drivers meeting Friday Dec 16
by Rodolfo Alonso Hernandez 14 Dec '22

14 Dec '22

Hello Neutrinos: Please check the Neutron drivers meeting agenda: https://wiki.openstack.org/wiki/Meetings/NeutronDrivers. So far we have two topics to discuss: * https://bugs.launchpad.net/neutron/+bug/1998609 * https://bugs.launchpad.net/neutron/+bug/1998608 See you next Friday.

1 0

[neutron] Neutron meetings cancelled last week of December
by Rodolfo Alonso Hernandez 14 Dec '22

14 Dec '22

Hello Neutrinos: The Neutron meetings (team, CI and drivers) will be cancelled the last week of December. Enjoy your holidays!

1 0

[nova][placement] Spec review day on Dec 14th
by Sylvain Bauza 14 Dec '22

14 Dec '22

Hey folks, As agreed on a previous meeting [1] we will have a second Spec review day on Wed Dec 14th. Now you're aware, prepare your specs in advance so they're reviewable and ideally please be around on that day in order to reply to any comments and eventually propose a new revision, ideally the same day. As a side note, we'll have an Implementation review day on Jan 10th which will focus on reviewing feature changes related to accepted specs and we also plan to review a few implementation patches by Dec 15th. Thanks, -Sylvain [1] https://meetings.opendev.org/meetings/nova/2022/nova.2022-11-29-16.00.log.h…

1 1

[all][tc] Technical Committee next weekly meeting on 2022 Dec 14 at 1600 UTC
by Ghanshyam Mann 14 Dec '22

14 Dec '22

Hello Everyone, The technical Committee's next weekly meeting is scheduled for 2022 Dec 14, at 1600 UTC. If you would like to add topics for discussion, please add them to the below wiki page by Tuesday, Dec 13 at 2100 UTC. https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting -gmann

1 1

[Keystone] member role with access rule can not list hypervisors details
by Hemant Sonawane 13 Dec '22

13 Dec '22

Hello, Does application credentials created as member role with access rule need any special permissions to list hypervisors via cli? Here is the access rule i created for my application credentials [ { "path": "/v2.1/**", "method": "GET", "service": "compute" } ] +--------------------------------+---------+--------+-------------------+ | ID | Service | Method | Path | +--------------------------------+---------+--------+-------------------+ | f2417b29f8394281aaa088d7c01059 | compute | POST | /v2.1/hypervisors | | 0cacd63becc04db086cd070db978c5 | compute | POST | /v2.1/servers | | 366abc32faf543b9ab047a83eafe83 | compute | GET | /v2.1/servers | | 4a4258b6a37c49f3816b708efa179f | compute | GET | /v2.1/** | +--------------------------------+---------+--------+-------------------+ I tried to use other access rule too. After sourcing the credentials if i would like to list any of the openstack resources it simply does not work *openstack volume list openstack The request you have made requires authentication. (HTTP 401)openstack hypervisor list The request you have made requires authentication. (HTTP 401) (Request-ID: req-c4532458-5606-4c85-8f93-20e61f57e2c9)* i think this could be a bug in keystone or it might need any special permissions with access rule. Please help will be really appreciated. -- Thanks and Regards, Hemant Sonawane

1 0

[cinder] volume_attachement entries are not getting deleted from DB
by Hemant Sonawane 13 Dec '22

13 Dec '22

Hello I am using wallaby release openstack and having issues with cinder volumes as once I try to delete, resize or unshelve the shelved vms the volume_attachement entries do not get deleted in cinder db and therefore the above mentioned operations fail every time. I have to delete these volume_attachement entries manually then it works. Is there any way to fix this issue ? nova-compute logs: cinderclient.exceptions.ClientException: Unable to update attachment.(Invalid volume: duplicate connectors detected on volume Help will be really appreciated Thanks ! -- Thanks and Regards, Hemant Sonawane

3 6

[barbican] Meeting cancelled today
by Grzegorz Grasza 13 Dec '22

13 Dec '22

Hi all, I won't be able to host the Barbican meeting today. As there are no outstanding topics to discuss, I'm cancelling it. / Greg

1 0

[all] many Python 3.11 failures in OpenStack components
by Thomas Goirand 13 Dec '22

13 Dec '22

Hi, The below bugs have been reported against the OpenStack components in Debian. I would greatly appreciate help fixing these before Bookworm is in freeze (in January). If you fix a bug, it'd be great to add a link in the Debian bug, so I can see the URL of the review (even if the patch isn't merged yet). As a reminder: to write to a Debian bug, simply write to <bugnumber>@bugs.debian.org (the Debian BTS is email driven). networking-mlnx: https://bugs.debian.org/1021924 FTBFS test failures: sqlalchemy.exc.InvalidRequestError: A transaction is already begun on this Session. networking-l2gw: https://bugs.debian.org/1023764 autopkgtest failure: module 'neutron.common.config' has no attribute 'register_common_config_options' ironic: https://bugs.debian.org/1024783 needs update for python3.11: Cannot spec a Mock object ironic-inspector: https://bugs.debian.org/1024952 needs update for python3.11: Failed to resolve the hostname (meow) for node uuid1 mistral-dashboard: https://bugs.debian.org/1024958 needs update for python3.11: module 'inspect' has no attribute 'getargspec'. Did you mean: 'getargs'? murano: https://bugs.debian.org/1025013 needs update for python3.11: 'NoneType' object has no attribute 'object_store' python-cinderclient: https://bugs.debian.org/1025028 needs update for python3.11: conflicting subparser: another-fake-action python-novaclient: https://bugs.debian.org/1025110 needs update for python3.11: conflicting subparser python-sushy: https://bugs.debian.org/1025124 needs update for python3.11: AssertionError: expected call not found python-taskflow: https://bugs.debian.org/1025126 needs update for python3.11: MismatchError Cheers, Thomas Goirand (zigo)

7 8

[neutron][devstack][qa] Dropping lib/neutron module
by Slawek Kaplonski 13 Dec '22

13 Dec '22

Hi, As You maybe know (or not as this was very long time ago) there are 2 modules to deploy Neutron in devstack: * old one called lib/neutron-legacy * new one called lib/neutron The problem is that new module lib/neutron was really never finished and used widely and still everyone is using (and should use) old one lib/neutron-legacy. We discussed that few times during PTGs and we finally decided to drop "new" module lib/neutron and have then rename old "lib/neutron-legacy" to be "lib/neutron" again. Decision was made because old module works fine and do its job, and as there is nobody who would like to finish really new module. Also having 2 modules from which "legacy" one is the only one which really works can be confusing and we want to avoid that confusion. So I proposed patches [1] and [2] to drop this unfinished module. I also proposed DNM patches for Neutron [3] and Tempest [4] to test that all jobs will work fine. But if You are maybe relaying on the Neutron modules from Devstack, please propose some test patch in Your project too and check if everything works fine for You. In patch [2] I didn't really removed "lib/neutron-legacy" yet because there are some projects which are sourcing that file in their devstack module. As [1] and [2] will be merged I will be proposing patches to change that for those projects but any help with that is welcome :) [1] https://review.opendev.org/c/openstack/devstack/+/865014 [2] https://review.opendev.org/c/openstack/devstack/+/865015 [3] https://review.opendev.org/c/openstack/neutron/+/865822 [4] https://review.opendev.org/c/openstack/tempest/+/865821 -- Slawek Kaplonski Principal Software Engineer Red Hat

2 1

[neutron][ironic] Distributed routers and SNAT
by Arnaud Morin 13 Dec '22

13 Dec '22

Hello team, When using router in DVR (+ HA), we end-up having the router on all computes where needed. So far, this is nice. We want to introduce Ironic baremetal servers, with a private network access. DVR won't apply on such baremetal servers, and we know floating IP are not going to work. Anyway, we were thinking that SNAT part would be OK. After doing few tests, we noticed that all distributed routers are answering to ARP and ICMP, thus creating duplicates in the network. $ arping -c1 192.168.43.1 ARPING 192.168.43.1 60 bytes from fa:16:3f:67:97:6a (192.168.43.1): index=0 time=634.700 usec 60 bytes from fa:16:3f:dc:67:91 (192.168.43.1): index=1 time=750.298 usec --- 192.168.43.1 statistics --- 1 packets transmitted, 2 packets received, 0% unanswered (1 extra) Is there anything possible on neutron side to prevent this? FYI, I did a comparison with routers in centralized mode (+ HA). In that situation, keepalived is putting the qr-xxx interface down in qrouter namespace. In distributed mode, keepalives is running in snat- namespace and cannot manage the router interface. Any help / tip would be appreciated. Thanks! Arnaud.

4 7

[neutron] Bug deputy report for week of December 5th
by Brian Haley 13 Dec '22

13 Dec '22

Hi, I was Neutron bug deputy last week. Below is a short summary about the reported bugs. -Brian Critical bugs ------------- * https://bugs.launchpad.net/neutron/+bug/1999249 - neutron-tempest-plugin jobs timing out on nested-virt nodes - Needs owner Medium bugs ----------- * https://bugs.launchpad.net/neutron/+bug/1998749 - [L3][DVR][vlan] east-west traffic flooding on physical bridges - Needs owner * https://bugs.launchpad.net/neutron/+bug/1998751 - [L3][DVR][vlan] HA VRRP traffic flooding on physical bridges on compute nodes - Needs owner * https://bugs.launchpad.net/neutron/+bug/1998952 - The static route removes the local route from qrouter-namespace - Adding/removing route same as attached interface subnet causes issue - Possible user error, the neutron API will let you "break" things - https://bugzilla.redhat.com/show_bug.cgi?id=1836870 has link to note - Needs discussion/owner * https://bugs.launchpad.net/neutron/+bug/1999154 - ovs/ovn source deployment broken with ovs_branch=master - https://review.opendev.org/c/openstack/neutron/+/867004 Low bugs -------- * https://bugs.launchpad.net/neutron/+bug/1998833 - [OVN] "MetadataAgentOvnSbIdl" should implement "post_connect" method - https://review.opendev.org/c/openstack/neutron/+/866601 Misc bugs --------- * https://bugs.launchpad.net/neutron/+bug/1999170 - Package names missing in neutron openvswitch setup guide - Added note that this information is in the install guide, closed * https://bugs.launchpad.net/neutron/+bug/1999209 - [ovn]dnat_adn_snat will not changed if updated fixed_ips of internal port - Asked for more information * https://bugs.launchpad.net/neutron/+bug/1999238 * https://bugs.launchpad.net/neutron/+bug/1999239 * https://bugs.launchpad.net/neutron/+bug/1999240 - Functional test failures - Related to https://review.opendev.org/c/openstack/neutron/+/867075

1 0

[openstack-ansible] Environment variable overflow on Ubuntu 20,04
by Taltavull Jean-François 12 Dec '22

12 Dec '22

Hello, The 'lxc_container_create' role appends the content of the 'global_environment_variables' OSA variable to '/etc/environment', which is used by the library 'libpam'. On Ubuntu 20.04 and 'libpam-runtime' library version 1.3.1, this works fine unless one of the environment variables content size exceeds 1024 char. In this case, the variable content is truncated. In our OpenStack deployment, this is the variable 'no_proxy' that overflows. This limit has been raised to 8192 with 'libpam-runtime' version 1.4 but there is no backport available in Ubuntu 20.04 repositories. So what would you think of the following workaround: instead of using '/etc/environment' to store the global variables, the role would create a shell script in '/etc/profile.d' thus avoiding the variable truncation issue related to the 'libpam' library ? This works on Ubuntu, Debian and CentOS hosts and containers. Cheers, JF

2 2

[Image Encryption] No meetings until next year
by Josephine Seifert 12 Dec '22

12 Dec '22

Hi, due to an injury and upcoming vacation, the next meeting for the image encrpyption team will be on Monday the 9th of Jannuary. greetings and happy holidays Luzi

2 1

Kolla-ansible [nova | Ensure RabbitMQ users exist] task error
by A Monster 12 Dec '22

12 Dec '22

I'm trying to deploy openstack xena on centos stream 8 on 2 controller nodes, 8 compute and a storage cluster, using kolla-ansible but I keep getting the following error, any workaround I could do to avoid this problem ? thank you. TASK [service-rabbitmq : nova | Ensure RabbitMQ users exist] ******************************************* skipping: [Controller-02] => (item=None) skipping: [Controller-02] FAILED - RETRYING: nova | Ensure RabbitMQ users exist (5 retries left). FAILED - RETRYING: nova | Ensure RabbitMQ users exist (4 retries left). FAILED - RETRYING: nova | Ensure RabbitMQ users exist (3 retries left). FAILED - RETRYING: nova | Ensure RabbitMQ users exist (2 retries left). FAILED - RETRYING: nova | Ensure RabbitMQ users exist (1 retries left). failed: [Controller-01 -> Controller-01] (item=None) => {"attempts": 5, "censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} fatal: [Controller-01 -> {{ service_rabbitmq_delegate_host }}]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}

2 1

auto switch of glance-api containers to other controllers
by pradeep 12 Dec '22

12 Dec '22

Hello All, I understand that glance-api container run only one controller always. I have tried to check if the glance-api container to switch over to the next available controller by means of rebooting and stopping the containers, but nothing happened. Is there a way to make sure that glance-api container switches to other controllers if controller 1 is not available. Thanks for you help Regards Pradeep

3 4

[ops] Migration from CentOS streams to Ubuntu and fast forward updates
by Massimo Sgaravatto 12 Dec '22

12 Dec '22

Dear all Dear all We are now running an OpenStack deployment: Yoga on CentOS8Stream. We are now thinking about a possible migration to Ubuntu for several reasons in particular: a- 5 years support for both the Operating System and OpenStack (considering LTS releases) b- Possibility do do a clean update between two Ubuntu LTS releases c- Easier procedure (also because of b) for fast forward updates (this is what we use to do) Considering the latter item, my understanding is that an update from Ubuntu 20.04 Ussuri to Ubuntu 22.04 Yoga could be done in the following way (we have two controller nodes and n compute nodes): - Update of first controller node from Ubuntu 20.04 Ussuri to Ubuntu 20.04 Victoria (update OpenStack packages + dbsync) - Update of first controller node from Ubuntu 20.04 Victoria to Ubuntu 20.04 Wallaby (update OpenStack packages + dbsync) - Update of first controller node from Ubuntu 20.04 Wallaby to Ubuntu 20.04 Xena (update OpenStack packages + dbsync) - Update of first controller node from Ubuntu 20.04 Xena to Ubuntu 20.04 Yoga (update OpenStack packages + dbsync) - Update of first controller node from Ubuntu 20.04 Yoga to Ubuntu 22.04 Yoga (update Ubuntu packages) - Update of second controller node from Ubuntu 20.04 Ussuri to Ubuntu 22.04 Yoga (update OpenStack and Ubuntu packages) - Update of the compute nodes from Ubuntu 20.04 Ussuri to Ubuntu 22.04 Yoga (update OpenStack and Ubuntu packages) We would do the same when migrating from Ubuntu 22.04 Yoga to Ubuntu 24.04 and the OpenStack xyz release (where xyz is the LTS release used in Ubuntu 24.04) Is this supposed to work or am I missing something ? If we decide to migrate to Ubuntu, the first step would be the reinstallation with Ubuntu 22.04/Yoga of each node currently running CentOS8 stream/Yoga. I suppose there are no problems having in the same OpenStack installation nodes running the same Openstack version but different operating systems, or am I wrong ? Thanks, Massimo

4 8

[dev][infra][qa][tact-sig] Tox 4.0.0 breaking changes
by Jeremy Stanley 12 Dec '22

12 Dec '22

Tox 4.0.0, a major rewrite, just appeared on PyPI in the past couple of hours, so jobs are breaking right and left. We're going to try pinning to latest 3.x in zuul-jobs for the near term while we work through some of the more obvious issues with it, but until then lots of tox-based Zuul jobs are going to be broken. I'll follow up to this thread once we think things should be stabilized and we can start trying to pick through what's going to need fixing for the new tox version. -- Jeremy Stanley

4 12

openstack 504 Gateway
by f.loghmani 11 Dec '22

11 Dec '22

hello I use Kolla ansible multi-region. in one of my regions that is not keystone and horizon on it, sometimes I receive the error Gateway Timeout. how could I solve this problem? (in the same time I could open other region panels and it works correctly) thanks in advance. Gateway Timeout The gateway did not receive a timely response from the upstream server or application

1 0