April 2024 - openstack-discuss - lists.openstack.org

[all][oslo][neutron][glance][ironic] Deprecating eventlet based WSGI server
by Takashi Kajinami 06 May '24

06 May '24

Hello, As you know there has been a long discussion about eventlet usage in OpenStack, and after seeing multiple problems caused by the library and no long term plan in the project, and we are moving toward doping the dependency on eventlet. https://review.opendev.org/c/openstack/governance/+/916546 One of the core implementation which heavily depends on eventlet is WSGI server. Multiple projects have been providing the -api console scripts (eg nova-api) based on the oslo.service's WSGI server implementation which depends on eventlet. On the other hand, nowadays we have multiple methods to launch api services, such as httpd+mod_wsgi, uwsgi and gunicorn. And a few projects in OpenStack don't provide the -api console script (eg. aodh, barbican and placement) but suggest using these external frameworks to run their api WSGI application. I know that multiple deployment tools (for example TripleO in the past) and some distros (Debian) already started using the external framework for consistency, better concurrency and so on. Given the situation, I wonder if we can consider deprecating the eventlet WSGI server in oslo.service in 2024.2/2025.1 and eventually remove all -api script in 2025.2. This was discussed during the previous nova PTG sessions and I think this makes complete sense as a global direction given the variety of alternative approaches currently available. I'm tagging neutron(*1) and glance(*2) because IIRC these had(or even have) some problems with running API in wsgi mode, and also ironic because I've seen some works coming from it, iiuc. Please let me know if you have any concern applying this approach to these (or the other) projects. Thank you, Takashi Kajinami (*1) For neutron, there are a few issues being fixed now https://bugs.launchpad.net/neutron/+bug/2062009 but ml2-ovn may need additional fixes. https://bugs.launchpad.net/neutron/+bug/1912359 (*2) As far as I know the issue with glance was already fixed some time ago[1]. https://bugs.launchpad.net/glance/+bug/1888713 -- Takashi Kajinami irc: tkajinam github: https://github.com/kajinamit launchpad: https://launchpad.net/~kajinamit

7 9

[all][packaging][release][qa] pep-517 and pip 23
by smooney＠redhat.com 06 May '24

06 May '24

hi folks, TL;DR ----- this is a long email so the tl;dr is we should add pyproject.toml ``` [build-system] requires = ["pbr>=5.7.0", "setuptools>=36.6.0", "wheel"] build-backend = "pbr.build" ``` to all openstack projects to supprot pip 23.1+ Long version ------------ As some of you may know the ptg happend last week an done of hte topic raised there was a know limitation in devstack where GLOBAL_VENV=True does not work on centos/fedora derived distros. while i generally dont use centos/fedora derived distros for my daily work i have been using devstack for over 10 years at this point so when im told it broken i ocationlly will go fix it. In this case the issue is not with devstack (it often isnt) and is actully in our collective reliance on a deprecated functionality and how we package openstack today. Back story, for better or worse when openstack was created the landscape of python packaging was significantly different then it is today. To help corodinate our dependcies the openstack comunity went all in on requirements.txt and developed upper-constraits to manage our requirements and developed PBR (python build Reasonableness). both of those approches have served us well and likely will continue to do so for some time. Since then pyproject.toml has started to gain popularity and pep-517 (A build-system independent format for source trees) has been adotpted but so far openstack as a comunity has not embraced those changes. (although pbr already supprot pep-517 :) ). Until now this has not been a problem however with recent release of pip things are now breaking. Problem Context: - openstack does not provide pyporject.toml files - openstack project configure setuptools to use pbr via setup.py - setuptools support generating scripts form entry points(console_scripts) which we use for project binaries( nova- compute, neutron-server, ...) - openstack api services use a pbr extension know as wsgi_scripts which adds a similar functionality for generating wsgi application mains file - devstack due to coming changes in distro to block "sudo pip install" now installs openstack in a single by default on debian distros. on centos/fedora distros it currently almost functional (but not). - devstack installs in editable mode because that is how most developers use devstack for developemnt What changed: - pip 20.2 deprecated support for falling back to calling setup.py when pyproject.toml is not present - https://pip.pypa.io/en/stable/news/#v20-2 - https://github.com/pypa/pip/issues/8368 - at this point when pyproject.yaml is not called pip calls setup.py install [-e] <path to repo or package name> - setup.py invokes pbr to generate wsgi_scripts and other files - in pip 23.1 the fallback was removed https://pip.pypa.io/en/stable/news/#v23-1 - pip now builds a temp local wheel instead of calling setup.py install when pyproject.toml is not found - in editable mode (-e) the default way the wheel is built does not invoke pbr properly to generate the wsgi files - as a result the wsgi files are not installed to the venv bin dir and this causes devstack to fail to start keystone (or any other wsgi api) The fix, there are a few ways to approch this: - clamp pip (https://review.opendev.org/c/openstack/devstack/+/898845) this is a terible idea and it breaks debian - clamping pip below 23.1 works on centos for now however OVS on debian already requires pep 517 for installing - as such when that version of ovs (or other projects start enforcing that) hits centos it will break - add a minimal pyproject.toml to all openstack project - example for keystone https://review.opendev.org/c/openstack/keystone/+/899519 - this example jsut adds the minium pyproject.toml to enable pbr to be used as the build system - it does not remove supprot for setup.py but it will not be used by default going forward whit new version of pip - all config is still in setup.cfg - add pyproject.toml support but remove setup.py support https://review.opendev.org/c/opendev/bindep/+/816741 - this goes one step futrue and actuly nulls out setup.py - we could take this approach if we want to but its not required. The reason for this email is to get agreement on how to support pip 23.1+ My proposal would be to add pyproject.toml with the minimal config required to enable pip 23.2 but not drop the setup.py support in this cycle. i would suggest that we can deprecate installing via setup.py in caracal if that is desirable but not make that change until the next slurp to give time to projects and packages to catch up. if there are no objections to that i would like to ask what is the preferd way to create and submit the patches for the minimal pyproject.toml ``` [build-system] requires = ["pbr>=5.7.0", "setuptools>=36.6.0", "wheel"] build-backend = "pbr.build" ``` i can propose patches to the main project deployed by devstack by default. i.e. nova/neutron ectra but it might be better if someone form the core teams of each project could take on that task and chasing the reviews. if someone has a script to generate a patch for a set of repos that just add the above pyporject.toml then we can use that to keep a consistent topic/commit message across all patches. my proposal for testing this is two fold. in ci we can test the patches are functional by using a DNM patch to devstack to force enable the gloal venv on centos and depend on the set of patches, i will modify https://review.opendev.org/c/openstack/devstack/+/898845 to do that locally we can test that the patch is functional by doing the following substituting keystone for the relevant project. ``` git clone https://opendev.org/openstack/keystone cd keystone python3 -m venv .venv . .venv/bin/activate python3 -m pip install -U pip python3 -m pip install -e . ls .venv/bin/keystone* ``` the ls will not contain the wsgi script with the addition of the pyproject.toml it should. on debian sid i can reproduce the same issue the problem we are seeing on centos 9 now will start impacting other distos eventually so its good to fix this now. if i start proposing this to glance/nova/neutron/cinder/swift/horizon/placement (to get a default devstack working) can i get cores to review? can cores form each project nominate peopel to review or volenteer to do this work. i have created https://etherpad.opendev.org/p/pep-517-and-pip-23 to track this work i have some other work that i think we can do in parallel like addign pyporject.toml support to the pti requirements if we go this way but that i think we can discuss on a governacne patch to updated it. anyway this is a lot to digest so feedback is welcome. if someone else would like to drive this that is also welcome (i.e. tc/release team) regards sean.

10 27

[nova][cinder] Openstack with FC SAN storage with HA
by Satish Patel 06 May '24

06 May '24

Folks, I have a question, Does openstack support FC SAN storage? I have deployed openstack with ceph but first time got this requirement that a customer has SAN storage because they were using VMware but now move to openstack. How does openstack talk to FC SAN and what about VM migration etc? Do I need to mount the LUN disk on each computer and point nova to the LUN disk? Or I have to create some kind of shared file system on top of the LUN and expose that to nova/cinder. Can someone explain the entire workflow?

5 8

Magnum error after Antelope upgrade
by Michel Jouvin 02 May '24

02 May '24

Hi, In a cloud where all services have been upgraded from Yoga to Antelope, I tried to upgrade Magnum but after the (successful) upgrade, Magnum reports "keystone client errors" when trying to retrieve the status of existing clusters or to create new ones (deletion of existing ones seem to work). The typical error is: ---- 2024-04-29 11:57:39.733 72464 WARNING magnum.service.periodic [-] Skip pulling data from cluster 2e8d6ee8-01cb-42d4-9b6d-0cf242cd4afa due to error: unexpected keystone clie nt error occurred: Forbidden (HTTP 403): magnum.common.exception.AuthorizationFailure: unexpected keystone client error occurred: Forbidden (HTTP 403) 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic Traceback (most recent call last): 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic File "/usr/lib/python3.9/site-packages/magnum/common/exception.py", line 57, in wrapped 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic return func(*args, **kw) 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic File "/usr/lib/python3.9/site-packages/magnum/common/clients.py", line 166, in barbican 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic self._barbican = barbicanclient.Client(session=session, 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic File "/usr/lib/python3.9/site-packages/barbicanclient/client.py", line 186, in Client 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic return client_class(session=session, *args, **kwargs) 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic File "/usr/lib/python3.9/site-packages/barbicanclient/v1/client.py", line 47, in __init__ 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic normalized_microversion = self._get_max_supported_version( 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic File "/usr/lib/python3.9/site-packages/barbicanclient/v1/client.py", line 88, in _get_max_supported_version 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic min_ver, max_ver = self._get_min_max_server_supported_microversion( 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic File "/usr/lib/python3.9/site-packages/barbicanclient/v1/client.py", line 119, in _get_min_max_server_supporte d_microversion 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic return self._get_min_max_version(session, endpoint, '1.1') 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic File "/usr/lib/python3.9/site-packages/barbicanclient/v1/client.py", line 125, in _get_min_max_version 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic resp = discover.get_version_data( 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic File "/usr/lib/python3.9/site-packages/keystoneauth1/discover.py", line 107, in get_version_data 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic resp = session.get(url, headers=headers, authenticated=authenticated) 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic File "/usr/lib/python3.9/site-packages/keystoneauth1/session.py", line 1141, in get 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic return self.request(url, 'GET', **kwargs) 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic File "/usr/lib/python3.9/site-packages/keystoneauth1/session.py", line 986, in request 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic raise exceptions.from_response(resp, method, url) 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic keystoneauth1.exceptions.http.Forbidden: Forbidden (HTTP 403) 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic --- Did I miss something in the upgrade procedure? Thanks in advance for any help. Best regards, Michel

4 9

Upgrade Bobcat/Caracal and python3-futurist
by Francesco Di Nucci 02 May '24

02 May '24

Hi all, I'm upgrading from Bobcat to Caracal using RDO repositories. I noticed that the package python3-futurist.noarch was present in centos-openstack-bobcat repo, but it is not in openstack-caracal. So it is not possible to update/reinstall/etc. it as it is "orphaned" once the bobcat repository is disabled. Should it be removed? Is it still a dependency? Best regards Francesco Di Nucci

4 6

[dev][requirements] Potentially ceasing development of Paste
by Jeremy Stanley 01 May '24

01 May '24

Just a heads up that Chris is raising concerns about continued maintenance of the Paste library, which more than a few of our projects rely on: https://github.com/cdent/paste/discussions/91 It's worth considering whether we would offer to adopt/fork it or if there are straightforward migration paths to actively maintained alternatives. -- Jeremy Stanley

3 3

[cinder] this week's meeting canceled
by Jon Bernard 30 Apr '24

30 Apr '24

Hello all, we will cancel tomorrow's regularly scheduled meeting in honor of International Workers' Day. See you all next week. -- Jon

1 0

Upgrade Bobcat/Caracal and python3-futurist
by Francesco Di Nucci 30 Apr '24

30 Apr '24

Hi all, I'm upgrading from Bobcat to Caracal using RDO repositories. I noticed that the package python3-futurist.noarch was present in centos-openstack-bobcat repo, but it is not in openstack-caracal. So it is not possible to update/reinstall/etc. it as it is "orphaned" once the bobcat repository is disabled. Should it be removed? Is it still a dependency? Best regards Francesco Di Nucci

1 0

[Octavia] next weekly meetings cancelled
by Gregory Thiemonge 30 Apr '24

30 Apr '24

Hi Folks, I'll be on Holiday for the 2 next weekly meetings (May 1st and 8th), so I'm cancelling them. The next meeting will be May 15th. For any topics (requesting reviews, questions, etc..) we're still reachable on IRC. Thanks Greg

1 0

Nova Antelope / RHEL9 : failure to lauch VM because video model virtio is not supported
by Michel Jouvin 30 Apr '24

30 Apr '24

Hi, After upgrading a compute server from Yoga/EL8 to Antelope/EL9, I cannot start VM on it. nova-compute log says: ---- Failed to build and run instance: libvirt.libvirtError: unsupported configuration: domain configuration does not support video model 'virtio' ---- According to https://bugs.launchpad.net/nova/+bug/2018172, it seems the problem also existed when trying to deploy nova-compute Yoga/Zed on EL9 but what fixed the problem is not clear in the issue... Any idea about what could cause this problem and how to fix it? Best regards, Michel

2 2