December 2023 - openstack-discuss

[kolla-ansible][zun] Intention to drop support for Zun for the 2023.1 and path forward
by Jan Gutter 21 Dec '23

21 Dec '23

Hi folks, In yesterday's Kolla IRC meeting we discussed the path forward for Zun [1] and came to the conclusion that it might be better for all involved to drop support for Zun for the 2023.1 release until it can be updated to support the necessary dependencies. More specific details at the end of the message. With this in mind, the plan is to start this process in a week from now, unless a better solution is presented. There is still some time for discussion, please don't hesitate! There are two paths forward after this: If Zun can be updated to support the dependencies during the maintenance timeframe of 2023.1, then support would be restored. If no support is available at the time Caracal releases, however, it will be considered for removal in kolla and kolla-ansible. What this means: Kolla-ansible operators currently running Zun should hold off migration to 2023.1. Any development help would of course be appreciated. Please coordinate with the Zun project to help with development. Details on the dependency problems: * Currently, Zun has an external dependency on Docker 20.x, and uses a feature that has been removed in later versions of Docker. * This Docker feature also relies on an older version of etcd (3.3) * The mitigation [2] is to pin to the old versions of Docker and etcd, but this is not possible with Debian Bookworm hosts. Why this is a problem in Kolla-ansible (and Kolla): * This can't easily be solved by vendoring in an older version of etcd: the migration path it introduces to operators would incur a significant amount of risk. * Host level involvement is also particularly painful, the host version of Docker needs to connect to the etcd service running in the containers. * Even though the containers can be kept buildable and the configuration and inventory settings could be maintained, a working setup is not testable in CI. Thanks very much for everyone's patience, contributions and time! [1]: https://meetings.opendev.org/meetings/kolla/2023/kolla.2023-09-13-13.00.log… [2]: https://bugs.launchpad.net/zun/+bug/2007142

2 2

[neutron] Neutron drivers meeting cancelled
by Rodolfo Alonso Hernandez 21 Dec '23

21 Dec '23

Hello Neutrinos: Due to the lack of agenda [1], tomorrow's meeting is cancelled. Have a nice weekend and happy holidays. [1]https://wiki.openstack.org/wiki/Meetings/NeutronDrivers

1 0

[nova] Next two weekly meetings CANCELLED
by Sylvain Bauza 21 Dec '23

21 Dec '23

That's it. See you back on Jan 9th :)

1 0

[Glance] Cancelling Today's Weekly Meeting
by Pranali Deore 21 Dec '23

21 Dec '23

Hello, Since most of the team members are not around and there is nothing much on agenda too, cancelling today's (21st Dec) weekly meeting. Thanks & Regards, Pranali Deore

1 0

Reg Openstack deployment
by Rohit Sansiya 21 Dec '23

21 Dec '23

Dear Sir, please help me ! I have a single compute node and i have to configure nova and neutron on that single compute node only. how to do this kindly help me sir. Thanks Your message is ready to be sent with the following file or link attachments: Rohit Sansiya https://sites.google.com/view/rpgcloud/research-india Note: To protect against computer viruses, e-mail programs may prevent sending or receiving certain types of file attachments. Check your e-mail security settings to determine how attachments are handled.

2 1

[i18n][infra] Future of translate.openstack.org
by Tony Breeds 21 Dec '23

21 Dec '23

Hello all, The infra team is in the process of replacing older bionic servers with jammy based ones. We're nearing the end of that process and one of the remaining servers is translate.openstack.org. This server is primarily running zanata, which if I understand correctly, is no longer in use by the i18n team. I understand that there is a new platform that requies some tooling, if a dedicated server is needed for that we can spin up a newer jammy based server. So can we just retire this translate.openstack.org? Yours Tony.

3 3

[all][tc][ptls] Eventlet: Python 3.12, and other concerning discoveries
by Jay Faulkner 21 Dec '23

21 Dec '23

Hi all, For those who aren't aware, eventlet (https://github.com/eventlet/eventlet) is a library that almost every OpenStack project depends on and uses heavily. It's the primary method in most OpenStack project for achieving any form of parallelism via greenthreading. A couple of weeks ago it was brought to my attention, in the #openstack-tc IRC channel, that the eventlet library was broken on python 3.12 due to changes to the SSL stdlib in python 3.12. I enlisted a G-Research Open Source Software (GR-OSS) developer from outside the OpenStack community, Itamar, to help dig into what would be required to get python 3.12 support for eventlet, and asked him to do an overall health check on the project. The results were extremely concerning, and are why I'm emailing the list today. First of all, the original concern -- python 3.12 support -- is being worked on right now; there is a PR up to migrate the SSL module here https://github.com/eventlet/eventlet/pull/817 and a PR up here: https://github.com/eventlet/eventlet/pull/820 to begin migrating from nose3, a deprecated testing framework used by eventlet which does not support python 3.12. It's worth noting that in the past (2020) https://github.com/eventlet/eventlet/issues/638 there was resistance to removing nose for testing due to at-the-time-desired python2.7 support. Even if they are able to be merged, these PRs only scratch the surface of the total effort needed to get eventlet working on python 3.12. Furthermore, activity on the eventlet project is extremely slim right now. There have been no commits to the primary branch in nearly eight months and it's been a single maintainer project for an extremely long time. There are also indications there may be external factors impacting the maintainers' availability. This lack of redundancy on a core library that we have no clear migration path off of is concerning as well. As a large community that uses eventlet; I hope we can improve it. Now we get to the cherry on top -- code quality and existing bugs. When Itamar began evaluating the overall state of the project, he found that approximately 80% of the unit tests were passing. Some of the failing unit tests were valid failures. In fact, one of the failure cases -- around locks not being properly greened on upgrade -- explains an Ironic database migrations unit test failure which we spent several weeks off-and-on debugging last cycle. Along these lines; although eventlet has CI configured; those tests have never passed: https://github.com/eventlet/eventlet/actions/workflows/test.yaml?query=is%3… . I don't have a specific proposal for a solution. I do know that right now, OpenStack services are built "all-in" on eventlet, with no obvious or easy migration path off. At least in the short term, keeping eventlet running well is critical to keeping OpenStack running well. I'm sending this email to raise an alarm about this situation, and to hope that community members can work together to solve the problem. Thanks for reading and your help, Jay Faulkner OpenStack TC Chair

8 15

[all] Python 3.12 issues and classes of bugs, helping me fixing them all...
by Thomas Goirand 20 Dec '23

20 Dec '23

Hi there! Since a few days, we have Python 3.12 as available interpreter version in Debian Unstable. All Debian packages were rebuilt by Lucas Neusbaumm, and about 60 Debian RC bugs were filled against the OpenStack related packages. You may have seen it in some of your projects, I was able to fix lots of bugs (so far: 28 bugs fixed since Wednesday morning, and counting...), and forwarded them to upstream OpenStack whenever possible: https://review.opendev.org/c/openstack/os-ken/+/903521 https://review.opendev.org/c/openstack/sahara-plugin-spark/+/903656 https://review.opendev.org/c/openstack/keystonemiddleware/+/903640 https://review.opendev.org/c/openstack/os-ken/+/903521 https://review.opendev.org/c/openstack/sahara-plugin-spark/+/903656 https://review.opendev.org/c/openstack/keystonemiddleware/+/903640 I'd appreciate reviews and merge for these, of course... Many reported issues were problems in a single package (like Eventlet, or taskflow not able to load asyncore), so some bugs were closed without any further action but an attempt of rebuilding the said package to check everything is as expected. However, for a lot of bugs, I do not know how to fix, which leads me to this message. Of course, mostly the hardest bugs are remaining now, as I fixes the most easy ones. Here's a few examples. In Barbican: https://bugs.debian.org/1058183 What is this "KeyError: slice(0, -1, None)" about? Many packages have the same issue (like networking-sfc, neutron-dynamic-routing, neutron-vpnaas, manila, rally and many others), so if I understand this class of bug, I'll be able to fix them all. Help would be very much appreciated on that one. In openstackclient: https://bugs.debian.org/1058097 The stackdump is: File "<frozen importlib._bootstrap>", line 1360, in _find_and_load File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 935, in _load_unlocked File "<frozen importlib._bootstrap>", line 1176, in exec_module File "<frozen codecs>", line 58, in <module> AttributeError: module 'sys' has no attribute 'byteorder' I don't really understand what's going on, but this smells like an issue in Python 3.12 itself. watcher: https://bugs.debian.org/1058377 looks like the interpreter segfauls when importing some modules in watcher.tests.api ?!? Mistral: https://bugs.debian.org/1058182 Manila: https://bugs.debian.org/1058182 AssertionError: Failed to wait for expected result: Execution f2a97f87-7b08-4171-819a-47e30a0a1840 must have reached state SUCCESS state but it is in RUNNING What's going on here? diskimage-builder: I was able to fix the "ModuleNotFoundError: No module named 'imp'" (that's trivial to fix, really...), but now I get: File "/usr/lib/python3/dist-packages/mock/mock.py", line 1245, in _importer thing = __import__(import_path) ^^^^^^^^^^^^^^^^^^^^^^^ ModuleNotFoundError: No module named 'growvols' looks like what I attempted to fix with rewriting a load_source (which is removed from Python 3.12) didn't work. python-pykmip: https://bugs.debian.org/1058411 That one feels tricky. wrap_socket is gone from the standard ssl module, though the example given in here: https://docs.python.org/3/library/ssl.html#ssl.SSLContext.wrap_socket is to use something like this: with context.wrap_socket(sock, server_hostname=hostname) as ssock: but that wont work with the way pykmip is written. So probably it needs some kind of rewrite? Looks like there's some commits upstream to address it, but no new release, unfortunately. I probably should attempt packaging the last Git commit. oslotest: https://bugs.debian.org/1058373 The broken test case attemps to disable the 'sys' module, but it looks like its failing to do so. Is there a change of behavior in the interpreter? wsgi-intercept: https://bugs.debian.org/1058378 TypeError: HTTPSConnection.__init__() got an unexpected keyword argument 'cert_file' No idea how to fix that one... :/ There's more which I probably didn't classify above. The full list of bugs is here: https://bugs.debian.org/cgi-bin/pkgreport.cgi?which=maint&data=team%2Bopens… Please ignore the first 5 though, which I don't really maintain and don't really care about (these aren't Py 3.12 issues). Many thanks to anyone that will help. Cheers, Thomas Goirand (zigo)

4 3

[Senlin][tc][all] Senlin project marked inactive by TC
by Jay Faulkner 20 Dec '23

20 Dec '23

OpenStack Community, The Technical Committee marked the Senlin project inactive[0] due to failing CI jobs and general lack of contribution. If you rely on Senlin, please take notice that without intervention, it will not be part of future OpenStack releases, including 2024.1 (Caracal). The only way for a project to leave inactive status is to become active again, and be removed from the inactive list, or to be retired from OpenStack. For more information, please see our documentation[1] on inactive projects. I encourage the OpenStack community to rally and support Senlin to bring it back into active status. If you are an operator and want more information on how to contribute and get this project going, please reply to this email and someone will be glad to help. Thank you, Jay Faulkner OpenStack TC Chair 0: https://review.opendev.org/c/openstack/governance/+/902626 1: https://governance.openstack.org/tc/reference/emerging-technology-and-inact…

1 0

[ec2-api][tc][all] ec2-api project marked inactive by TC
by Jay Faulkner 20 Dec '23

20 Dec '23

OpenStack Community, The Technical Committee marked the ec2-api project inactive[0] due to failing CI jobs and general lack of contribution. If you rely on ec2-api, please take notice that without intervention, it will not be part of future OpenStack releases, including 2024.1 (Caracal). The only way for a project to leave inactive status is to become active again, and be removed from the inactive list, or to be retired from OpenStack. For more information, please see our documentation[1] on inactive projects. I encourage the OpenStack community to rally and support ec2-api to bring it back into active status. If you are an operator and want more information on how to contribute and get this project going, please reply to this email and someone will be glad to help. Thank you, Jay Faulkner OpenStack TC Chair 0: https://review.opendev.org/c/openstack/governance/+/902625 1: https://governance.openstack.org/tc/reference/emerging-technology-and-inact…

1 0