November 2023 - openstack-discuss

ERROR openstack [-] Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat.| Openstack tripleo wallaby version
by Swogat Pradhan 24 Dec '24

24 Dec '24

Hi, I am currently trying to deploy openstack wallaby using tripleo arch. I created the network jinja templates, ran the following commands also: #openstack overcloud network provision --stack overcloud --output networks-deployed-environment.yaml custom_network_data.yaml # openstack overcloud network vip provision --stack overcloud --output vip-deployed-environment.yaml custom_vip_data.yaml # openstack overcloud node provision --stack overcloud --overcloud-ssh-key /home/stack/sshkey/id_rsa overcloud-baremetal-deploy.yaml and used the environment files in the openstack overcloud deploy command: (undercloud) [stack@hkg2director ~]$ cat deploy.sh #!/bin/bash THT=/usr/share/openstack-tripleo-heat-templates/ CNF=/home/stack/ openstack overcloud deploy --templates $THT \ -r $CNF/templates/roles_data.yaml \ -n $CNF/workplace/custom_network_data.yaml \ -e ~/containers-prepare-parameter.yaml \ -e $CNF/templates/node-info.yaml \ -e $CNF/templates/scheduler-hints.yaml \ -e $CNF/workplace/networks-deployed-environment.yaml \ -e $CNF/workplace/vip-deployed-environment.yaml \ -e $CNF/workplace/overcloud-baremetal-deployed.yaml \ -e $CNF/workplace/custom-net-bond-with-vlans.yaml Now when i run the ./deploy.sh script i encounter an error stating: ERROR openstack [-] Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat. The generated environments from 'openstack overcloud baremetal provision' and 'openstack overcloud network provision' must be included with the deployment command.: tripleoclient.exceptions.InvalidConfiguration: Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat. The generated environments from 'openstack overcloud baremetal provision' and 'openstack overcloud network provision' must be included with the deployment command. 2022-04-19 13:47:16.582 735924 INFO osc_lib.shell [-] END return value: 1 Can someone tell me where the mistake is? With regards, Swogat Pradhan

4 4

[kolla] [openeuler] CI jobs status
by Michał Nasiadka 26 Nov '24

26 Nov '24

Hello Koalas, OpenEuler jobs in master (to be Antelope) are failing for a long time. If no one is interested in maintaining OpenEuler support in Kolla-Ansible - I will propose dropping it’s support on Wednesday Kolla meeting. Best regards, Michal

4 3

Kolla Ansible on Ubuntu 20.04 - cloud-init & other network issues
by Tobias McNulty 10 Oct '24

10 Oct '24

Hi, I'm attempting to use Kolla Ansible 14.6.0 to deploy OpenStack Yoga on a small 3-node Ubuntu 20.04 cluster. The nodes have 128 GB RAM each, dual Xeon processors, and dual 10G Intel NICs. The NICs are connected to access ports on a 10G switch with separate VLANs for the local and external networks. All the playbooks run cleanly, but cloud-init is failing in the Ubuntu 20.04 and 22.04 VMs I attempt to boot. The VM images are unmodified from https://cloud-images.ubuntu.com/, and cloud-init works fine if I mount a second volume with user-data. The error is a timeout attempting to reach 169.254.169.254. This occurs both when booting a VM in an internal routed network and directly in an external network. I tried various neutron plugin agents (ovn, linuxbridge, and openvswitch both with and without firewall_driver = openvswitch <https://docs.openstack.org/kolla-ansible/latest/reference/networking/neutro…>) first with a clean install of the entire OS each time, all with the same result. Running tcpdump looking for 169.254.169.254 shows nothing. As a possible clue, the virtual NICs are unable to pass any traffic (e.g., to reach an external DHCP server) unless I completely disable port security on the interface (even if the associated security group is wide open). But disabling port security does not fix cloud-init (not to mention I don't really want to disable port security). Are there any additional requirements related to deploying OpenStack with Kolla on Ubuntu 20.04? This is a fairly vanilla configuration using the multinode inventory as a starting point. I tried to follow the Quick Start <https://docs.openstack.org/kolla-ansible/yoga/user/quickstart.html> as closely as possible; the only material difference I see is that I'm using the same 3 nodes for control + compute. I am using MAAS so it's easy to get a clean OS install on all three nodes ahead of each attempt. I plan to try again with the standard (non-HWE) kernel just in case, but otherwise I am running out of ideas. In case of any additional clues, here are my globals.yml and inventory file, along with the playbook I'm using to configure the network, images, VMs, etc., after bootstrapping the cluster: https://gist.github.com/tobiasmcnulty/7dbbdbc67abc08cbb013bf5983852ed6 Thank you in advance for any advice! Cheers, Tobias

5 9

Live migration fails
by Szabo, Istvan (Agoda) 07 Aug '24

07 Aug '24

Hi, I have couple of compute nodes where the live migration fails with existing vms. When I quickly spawn a vm and try live migration it works so I assume shouldn't be a big problem with the compute node. However I have many existing vms where it fails with a servername not found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 ERROR nova.conductor.tasks.migrate [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] [instance: 1517a2ac-3b51-4d8d-80b3-89a5614d1ae0] Unable to find record for source node servername on servername: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 WARNING nova.scheduler.utils [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] Failed to compute_task_migrate_server: Compute host servername could not be found.: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 WARNING nova.scheduler.utils [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] [instance: 1517a2ac-3b51-4d8d-80b3-89a5614d1ae0] Setting instance to ACTIVE state.: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.672 227612 ERROR oslo_messaging.rpc.server [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] Exception during message handling: ComputeHostNotFound: Compute host am-osfecn-4025 Tried with this command: nova live-migration --block-migrate id. Any idea? Thank you. ________________________________ This message is confidential and is for the sole use of the intended recipient(s). It may also be privileged or otherwise protected by copyright or other legal rules. If you have received it by mistake please let us know by reply email and delete it from your system. It is prohibited to copy this message or disclose its content to anyone. Any confidentiality or privilege is not waived or lost by any mistaken delivery or unauthorized disclosure of the message. All messages sent to and from Agoda may be monitored to ensure compliance with company policies, to protect the company's interests and to remove potential malware. Electronic messages may be intercepted, amended, lost or deleted, or contain viruses.

5 16

[ptl][tc] OpenStack packages PyPi additional external maintainers audit & cleanup
by Ghanshyam Mann 17 May '24

17 May '24

Hi PTLs, As you might know or have seen for your project package on PyPi, OpenStack deliverables on PyPi have additional maintainers, For example, https://pypi.org/project/murano/, https://pypi.org/project/glance/ We should keep only 'openstackci' as a maintainer in PyPi so that releases of OpenStack deliverables can be managed in a single place. Otherwise, we might face the two sets of maintainers' places and packages might get released in PyPi by additional maintainers without the OpenStack project team knowing about it. One such case is in Horizon repo 'xstatic-font-awesome' where a new maintainer is added by an existing additional maintainer and this package was released without the Horizon team knowing about the changes and release. - https://github.com/openstack/xstatic-font-awesome/pull/2 To avoid the 'xstatic-font-awesome' case for other packages, TC discussed it in their weekly meetings[1] and agreed to audit all the OpenStack packages and then clean up the additional maintainers in PyPi (keep only 'openstackci' as maintainers). To help in this task, TC requests project PTL to perform the audit for their project's repo and add comments in the below etherpad. - https://etherpad.opendev.org/p/openstack-pypi-maintainers-cleanup Thanks to knikolla to automate the listing of the OpenStack packages with additional maintainers in PyPi which you can find the result in output.txt at the bottom of this link. I have added the project list of who needs to check their repo in etherpad. - https://gist.github.com/knikolla/7303a65a5ddaa2be553fc6e54619a7a1 Please complete the audit for your project before March 15 so that TC can discuss the next step in vPTG. [1] https://meetings.opendev.org/meetings/tc/2023/tc.2023-01-11-16.00.log.html#… -gmann

18 27

[all][packaging][release][qa] pep-517 and pip 23
by smooney＠redhat.com 06 May '24

06 May '24

hi folks, TL;DR ----- this is a long email so the tl;dr is we should add pyproject.toml ``` [build-system] requires = ["pbr>=5.7.0", "setuptools>=36.6.0", "wheel"] build-backend = "pbr.build" ``` to all openstack projects to supprot pip 23.1+ Long version ------------ As some of you may know the ptg happend last week an done of hte topic raised there was a know limitation in devstack where GLOBAL_VENV=True does not work on centos/fedora derived distros. while i generally dont use centos/fedora derived distros for my daily work i have been using devstack for over 10 years at this point so when im told it broken i ocationlly will go fix it. In this case the issue is not with devstack (it often isnt) and is actully in our collective reliance on a deprecated functionality and how we package openstack today. Back story, for better or worse when openstack was created the landscape of python packaging was significantly different then it is today. To help corodinate our dependcies the openstack comunity went all in on requirements.txt and developed upper-constraits to manage our requirements and developed PBR (python build Reasonableness). both of those approches have served us well and likely will continue to do so for some time. Since then pyproject.toml has started to gain popularity and pep-517 (A build-system independent format for source trees) has been adotpted but so far openstack as a comunity has not embraced those changes. (although pbr already supprot pep-517 :) ). Until now this has not been a problem however with recent release of pip things are now breaking. Problem Context: - openstack does not provide pyporject.toml files - openstack project configure setuptools to use pbr via setup.py - setuptools support generating scripts form entry points(console_scripts) which we use for project binaries( nova- compute, neutron-server, ...) - openstack api services use a pbr extension know as wsgi_scripts which adds a similar functionality for generating wsgi application mains file - devstack due to coming changes in distro to block "sudo pip install" now installs openstack in a single by default on debian distros. on centos/fedora distros it currently almost functional (but not). - devstack installs in editable mode because that is how most developers use devstack for developemnt What changed: - pip 20.2 deprecated support for falling back to calling setup.py when pyproject.toml is not present - https://pip.pypa.io/en/stable/news/#v20-2 - https://github.com/pypa/pip/issues/8368 - at this point when pyproject.yaml is not called pip calls setup.py install [-e] <path to repo or package name> - setup.py invokes pbr to generate wsgi_scripts and other files - in pip 23.1 the fallback was removed https://pip.pypa.io/en/stable/news/#v23-1 - pip now builds a temp local wheel instead of calling setup.py install when pyproject.toml is not found - in editable mode (-e) the default way the wheel is built does not invoke pbr properly to generate the wsgi files - as a result the wsgi files are not installed to the venv bin dir and this causes devstack to fail to start keystone (or any other wsgi api) The fix, there are a few ways to approch this: - clamp pip (https://review.opendev.org/c/openstack/devstack/+/898845) this is a terible idea and it breaks debian - clamping pip below 23.1 works on centos for now however OVS on debian already requires pep 517 for installing - as such when that version of ovs (or other projects start enforcing that) hits centos it will break - add a minimal pyproject.toml to all openstack project - example for keystone https://review.opendev.org/c/openstack/keystone/+/899519 - this example jsut adds the minium pyproject.toml to enable pbr to be used as the build system - it does not remove supprot for setup.py but it will not be used by default going forward whit new version of pip - all config is still in setup.cfg - add pyproject.toml support but remove setup.py support https://review.opendev.org/c/opendev/bindep/+/816741 - this goes one step futrue and actuly nulls out setup.py - we could take this approach if we want to but its not required. The reason for this email is to get agreement on how to support pip 23.1+ My proposal would be to add pyproject.toml with the minimal config required to enable pip 23.2 but not drop the setup.py support in this cycle. i would suggest that we can deprecate installing via setup.py in caracal if that is desirable but not make that change until the next slurp to give time to projects and packages to catch up. if there are no objections to that i would like to ask what is the preferd way to create and submit the patches for the minimal pyproject.toml ``` [build-system] requires = ["pbr>=5.7.0", "setuptools>=36.6.0", "wheel"] build-backend = "pbr.build" ``` i can propose patches to the main project deployed by devstack by default. i.e. nova/neutron ectra but it might be better if someone form the core teams of each project could take on that task and chasing the reviews. if someone has a script to generate a patch for a set of repos that just add the above pyporject.toml then we can use that to keep a consistent topic/commit message across all patches. my proposal for testing this is two fold. in ci we can test the patches are functional by using a DNM patch to devstack to force enable the gloal venv on centos and depend on the set of patches, i will modify https://review.opendev.org/c/openstack/devstack/+/898845 to do that locally we can test that the patch is functional by doing the following substituting keystone for the relevant project. ``` git clone https://opendev.org/openstack/keystone cd keystone python3 -m venv .venv . .venv/bin/activate python3 -m pip install -U pip python3 -m pip install -e . ls .venv/bin/keystone* ``` the ls will not contain the wsgi script with the addition of the pyproject.toml it should. on debian sid i can reproduce the same issue the problem we are seeing on centos 9 now will start impacting other distos eventually so its good to fix this now. if i start proposing this to glance/nova/neutron/cinder/swift/horizon/placement (to get a default devstack working) can i get cores to review? can cores form each project nominate peopel to review or volenteer to do this work. i have created https://etherpad.opendev.org/p/pep-517-and-pip-23 to track this work i have some other work that i think we can do in parallel like addign pyporject.toml support to the pti requirements if we go this way but that i think we can discuss on a governacne patch to updated it. anyway this is a lot to digest so feedback is welcome. if someone else would like to drive this that is also welcome (i.e. tc/release team) regards sean.

10 27

Accessing virsh in openstack wallaby | tripleo
by Swogat Pradhan 24 Mar '24

24 Mar '24

Hi, Can someone please tell me how can i access the virsh utility on the compute node on openstack wallaby? Previously we used nova_libvirt but in wallaby there are multiple containers and i am unable to use virsh utility to edit my domain.xml file. List of containers: [root@dcn01-hci-2 ~]# podman ps | grep nova 10d981158842 172.25.201.68:8787/tripleomaster/openstack-nova-libvirt:current-tripleo kolla_start 2 months ago Up 2 months ago nova_virtlogd_wrapper e91667516afc 172.25.201.68:8787/tripleomaster/openstack-nova-libvirt:current-tripleo kolla_start 2 months ago Up 2 months ago nova_virtsecretd a4b07fe0e833 172.25.201.68:8787/tripleomaster/openstack-nova-libvirt:current-tripleo kolla_start 2 months ago Up 2 months ago nova_virtnodedevd d7e1db393e9c 172.25.201.68:8787/tripleomaster/openstack-nova-libvirt:current-tripleo kolla_start 2 months ago Up 2 months ago nova_virtstoraged 5308a171793e 172.25.201.68:8787/tripleomaster/openstack-nova-libvirt:current-tripleo kolla_start 2 months ago Up 2 months ago nova_virtqemud f490a6249ba1 172.25.201.68:8787/tripleomaster/openstack-nova-libvirt:current-tripleo kolla_start 2 months ago Up 6 weeks ago nova_virtproxyd ab0441e56957 172.25.201.68:8787/tripleomaster/openstack-nova-compute:current-tripleo kolla_start 2 months ago Up 2 months ago (healthy) nova_migration_target f35fbba5b690 172.25.201.68:8787/tripleomaster/openstack-nova-compute:current-tripleo kolla_start 2 months ago Up 6 weeks ago (unhealthy) nova_compute 82bc01993106 172.25.201.68:8787/tripleomaster/openstack-nova-libvirt:current-tripleo /usr/sbin/virtlog... 2 months ago Up 2 months ago Can someone please suggest how I can i edit an instance xml file in this setup? With regards, Swogat Pradhan

2 5

[Freezer] Is Freezer dead?
by pi3.14＠tuta.io 19 Mar '24

19 Mar '24

Hi folks, is Freezer still supported? I don't see any real development going on with it. Is it recommended to use in a production Cloud environment? Are there other free and open source alternatives to Freezer? Thanks Roby

8 19

[cinder][glance][nova][telemetry][monasca][oslo][vmware] Can we deprecate(and remove) VMWare support ?
by Takashi Kajinami 07 Mar '24

07 Mar '24

Hi all, As you might know, some of the OpenStack projects currently support usage of VMWare component. However recently the VMware virt driver in nova was marked experimental in 2023.2 release [1] and it has been discussed that the driver will be removed in 2023.2 release[2], after deprecation is backported to 2023.1. (Please someone from nova team correct me if I'm wrong) So my current question is; Does it make sense to deprecate all of the features to support VMWare component ? My understanding is that these features were added to be used along with VMWare virt driver in nova, but once the virt driver is removed this use case is voided. I'll list up the features within the discussion scope below. nova: VMWare virt driver (this is already experimental by [1]) cinder: VMware Storage Driver (vmdk) glance: VMWare Datastore (I proposed deprecating this in [3][4]) ceilometer: VMWare virt driver (this was deprecated by [5]) os-brick: VmdkConnector monasca-agent: vcenter collector (*1) oslo.vmware: the whole project NOTE: These features are found based on dependency on oslo.vmware but there can be a few not listed here. Outside of OpenStack's governance, there are two projects dependent on oslo.vmware, networking-vsphere and vmware-nsx but both of these look inactive. Because of SLURP, we have to complete deprecation during in 2024.1 release so that we can drop the feature right after 2024.1. If we fail then we have to maintain these features until 2025.1 (the features already deprecated can be removed earlier, though). So I'd like to get feedback and decision before deadline for 2024.1 Thank you, Takashi (*1) Monasca is now inactive so I doubt anyone would work on this topic before it is retired. [1] https://review.opendev.org/c/openstack/nova/+/863911 [2] https://review.opendev.org/c/openstack/nova/+/897017 [3] https://review.opendev.org/c/openstack/glance_store/+/893696 [4] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [5] https://review.opendev.org/c/openstack/ceilometer/+/894540

6 21

[Watcher] How use Watcher to scheduler Openstack resource without cpu_usage
by Nguyễn Hữu Khôi 06 Mar '24

06 Mar '24

Hello guys. I want to use watcher to scheduler openstack resources but some openstack versions are missing cpu_usage metric. How do we use Watcher without it? Thank you. Nguyen Huu Khoi

3 5

[openstack][skyline] Some works with skyline
by Nguyễn Hữu Khôi 26 Feb '24

26 Feb '24

Hello guys. I am working on Skyline. I can integrate my monitor tab on it. I will try to do something about it. If it works properly, I can contribute. It is a very nice project. I hope it will grow and grow in the future. Thanks Skyline Team. [image: image.png] Nguyen Huu Khoi

3 7

[neutron][vpnaas][ovn] Pushing VPNaaS support through OVN
by Mohammed Naser 12 Feb '24

12 Feb '24

Hi all! One of the biggest show stoppers for us for OVN is VPNaaS but I think it’s time that we’d like to be involved in getting it over the line, I want to thank Bodo who has shown a lot of patience in getting the code necessary, and it seems that the code works for them in production as well. I’d like to ask if possible for folks to review these two outstanding changes: https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/847007 https://review.opendev.org/c/openstack/neutron-vpnaas/+/765353 I know Slawek had some concerns, so hoping that raising via the email list if Bodo can address those and over the next bit of time we can get this landed in, since it even has actual proper testing both in CI and with user feedback. Thanks! Mohammed

6 8

about magnum capi for production
by Nguyễn Hữu Khôi 07 Feb '24

07 Feb '24

Hello guys. Could you share with me some experience about using Magnum CAPI in production? https://github.com/vexxhost/magnum-cluster-api Thank you so much. Nguyen Huu Khoi

6 44

[all] SQLAlchemy 2.x support in Bobcat
by Thomas Goirand 02 Feb '24

02 Feb '24

Hi, As you may know, and to my great frustration, I'm not the maintainer of SQLAlchemy in Debian, even though OpenStack is the biggest consumer of it. The current maintainer insists that he wants to upload SQLA 2.x in Unstable, potentially breaking all of OpenStack. At the present moment, if I understand correctly, we're not there yet, and Bobcat doesn't have such a support. It would be ok for me, *IF* there are patches available on master, that I could backport to Bobcat and maintain in the debian/patches folder of each project. However, the biggest current annoyance, is that I have no idea where we are at. Are we close to such a support? Is there a list of patches to apply on top of Bobcat that is maintained somewhere? Please enlighten me... :) Cheers, Thomas Goirand (zigo)

5 10

[murano][heat][mistral][tc] Maintenance of yaql library
by Takashi Kajinami 21 Jan '24

21 Jan '24

Hello, The yaql library is currently maintained under murano's governance and is used by not only murano but a few other projects such as heat and mistral. In heat the library is essential since we introduced yaql function support. Looking at gerrit groups, I noticed the yaql library is maintained by a very small core group[1] and release group[2], but I'm afraid most of the people in these groups are no longer active in the community. Because heat is dependent on the library, I'm wondering if some of the heat cores can be added to these two groups so that we (at least I) can help maintaining the library. I've checked the changes merged into murano repo for these two years but iiuc all of these changes are gate fix. Also a dummy change[3] indicates that its CI is currently broken. If the murano project is no longer active then I'm also willing to take over yaql and bring it to heat's governance (of cause only if it's acceptable). I'll add the current PTL as cc, but please let me know if anyone has any opinion or concern. # I'll add mistral tag because the project is also dependent on yaql. # Also tc is added because this discussion is related to project governance. Thank you Takashi Kajinami [1] https://review.opendev.org/admin/groups/cba4f6fd54c3cc0d5706fc50b5ebd758860… [2] https://review.opendev.org/admin/groups/d5e894b34e5ab5a9fc2938a0d4e7bc76fd2… [3] https://review.opendev.org/c/openstack/murano/+/901888

3 7

[nova][release][tooz] Hold your rechecks
by Sylvain Bauza 10 Jan '24

10 Jan '24

Hey folks, FWIW, our grenade check on SLURP upgrades is 100% failing now [1] due to a behavioural change in the tooz library [2] that's pulled by Cinder and which was released [3] and delivered to upper-contraints yesterday [4] There are different possibilities in order to unblock the gate, but until one of them is agreed and merged, please avoid wasting our CI resources by asking to recheck for something which will fail. Thanks, -Sylvain (who will send an email once the job is back up) [1] https://zuul.opendev.org/t/openstack/builds?job_name=grenade-skip-level-alw… [2] https://review.opendev.org/c/openstack/tooz/+/891355 [3] https://review.opendev.org/c/openstack/releases/+/899966 [4] https://review.opendev.org/c/openstack/requirements/+/901131

4 6

[designate] Proposal to deprecate the agent framework and agent based backends
by Michael Johnson 09 Jan '24

09 Jan '24

TLDR: The Designate team would like to deprecate the backend agent framework and the agent based backends due to lack of development and design issues with the current implementation. The following backends would be deprecated: Bind9 (Agent), Denominator, Microsoft DNS (Agent), Djbdns (Agent), Gdnsd (Agent), and Knot2 (Agent). Designate includes many backend DNS server drivers[1], many of which are "native" (also known as xfr type backends) backend implementations. In addition to the "native" backends, Designate has an agent backend[2] that supports other backends via an agent process. To quote the agent backend documentation[2]: This backend uses an extension[3] of the DNS protocol itself to send management requests to the remote agent processes, where the requests will be actioned. The rpc traffic between designate and the agent is both unauthenticated and unencrypted. Do not run this traffic over unsecured networks. Here are the reasons we are proposing to deprecate the agent framework now: 1. The agent protocol used by Designate is using an "unassigned"[4][5] DNS opcode (14) that is causing problems with the dnspython library >= 2.3.0 which is now validating the opcode when building DNS messages. It is a bad practice to use "unassigned" values as they may be officially assigned at any time, likely with an incompatible message format. 2. The agent backends are not tested in the OpenStack jobs[1]. 3. Many of the agent backends have been marked as "Experimental" since 2016 with no additional contributions beyond general repository code maintenance. 4. The protocol between the Designate worker process and the agent process is unauthenticated and unencrypted. 5. We do not know of a development resource to rewrite the agent framework protocol to address issue #1 and #4 above. 6. The introduction of catalog zones[6] may eliminate the need for some of the agent based backend drivers. By marking the agent framework and agent based backends "deprecated" in the Antelope cycle, we would remove the code no earlier than the "C" release of OpenStack (per the OpenStack deprecation policy[7]). In the meantime, issue #1 has been worked around by overriding the dnspython opcode validation[7] as needed in the Designate code (similar to a monkey patch). This is not a sustainable long term solution. The following backend agent based drivers would be marked "deprecated" in addition to the agent framework itself: Bind9 (Agent) Denominator Microsoft DNS (Agent) Djbdns (Agent) Gdnsd (Agent) Knot2 (Agent) We plan to propose patches for the deprecation over the next week, but will not merge them until at least January 24th to allow time for comment from the community. If you have concerns about this deprecation plan or are interested in rewriting the agent framework protocol to address the above issues, please reply to this announcement. Michael [1] https://docs.openstack.org/designate/latest/admin/support-matrix.html [2] https://docs.openstack.org/designate/latest/admin/backends/agent.html [3] https://github.com/openstack/designate/blob/master/designate/backend/privat… [4] https://www.rfc-editor.org/rfc/rfc6895.html#section-2.2 [5] https://www.iana.org/assignments/dns-parameters/dns-parameters.xhtml#dns-pa… [6] https://www.ietf.org/archive/id/draft-ietf-dnsop-dns-catalog-zones-08.txt [7] https://docs.openstack.org/project-team-guide/deprecation.html [8] https://review.opendev.org/c/openstack/designate/+/870678

3 13

[TC][Monasca] Proposal to mark Monasca as an inactive project
by Sławek Kapłoński 05 Jan '24

05 Jan '24

Hi, I just proposed patch [1] to mark Monasca as inactive project. It was discussed during last TC meeting on 03.10.2023. Reasons for that are described in the commit message of the [1] but I will also put all of it here: previous cycle there wasn't almost any contributions to that project and most active contributors in last 180 days were Elod Illes and Dr. Jens Harbott who were fixing some Zuul configuration issues only. Here are detailed statistics about Monasca projects: Validating Gerrit... * There are 7 ready for review patches generated within 180 days * There are 2 not reviewed patches generated within 180 days * There are 5 merged patches generated within 180 days * Unreviewed patch rate for patches generated within 180 days is 28.0 % * Merged patch rate for patches generated within 180 days is 71.0 % * Here's top 10 owner for patches generated within 180 days (Name/Account_ID: Percentage): - Dr. Jens Harbott : 42.86% - Joel Capitao : 28.57% - Elod Illes : 28.57% Validate Zuul... Set buildsets fetch size to 500 * Repo: openstack/monasca-log-api gate job builds success rate: 82% * Repo: openstack/monasca-statsd gate job builds success rate: 95% * Repo: openstack/monasca-tempest-plugin gate job builds success rate: 81% * Repo: openstack/monasca-common gate job builds success rate: 84% * Repo: openstack/monasca-kibana-plugin gate job builds success rate: 83% * Repo: openstack/monasca-ceilometer gate job builds success rate: 100% * Repo: openstack/monasca-events-api gate job builds success rate: 83% * Repo: openstack/monasca-ui gate job builds success rate: 88% * Repo: openstack/monasca-specs gate job builds success rate: 100% * Repo: openstack/monasca-grafana-datasource gate job builds success rate: 100% * Repo: openstack/monasca-persister gate job builds success rate: 98% * Repo: openstack/monasca-notification gate job builds success rate: 93% * Repo: openstack/monasca-thresh gate job builds success rate: 100% * Repo: openstack/monasca-api gate job builds success rate: 76% * Repo: openstack/monasca-agent gate job builds success rate: 98% * Repo: openstack/python-monascaclient gate job builds success rate: 100% * Repo: openstack/monasca-transform gate job builds success rate: 100% What's next? According to the "Emerging and inactive projects" document [2] if there will be no volunteers who will step up and want to maintain this project before Milestone-2 of the 2024.1 cycle (week of Jan 08 2024) there will be no new release of Monasca in the 2024.1 cycle and TC will discuss if project should be retired. So if You are interested in keeping Monasca alive and active, please reach out to the TC to discuss that ASAP. Thx in advance. [1] https://review.opendev.org/c/openstack/governance/+/897520 [2] https://governance.openstack.org/tc/reference/emerging-technology-and-inact… -- Slawek Kaplonski Principal Software Engineer Red Hat

8 14

[openstack][kolla-ansible] kolla-ansible stucked when reconfigure
by Nguyễn Hữu Khôi 22 Dec '23

22 Dec '23

Hello guys. I am operating an openstack cluster with kolla but sometimes I cannot reconfigure. It stucked at some hosts(gather fact) and I need to restart docker service on the stucked host to continue. I tried to google but I cannot find the root cause. Would you share some experiences on it because my solution causes instances on host to timeout some seconds. Thank you. Nguyen Huu Khoi

2 7

[kolla-ansible][zun] Intention to drop support for Zun for the 2023.1 and path forward
by Jan Gutter 21 Dec '23

21 Dec '23

Hi folks, In yesterday's Kolla IRC meeting we discussed the path forward for Zun [1] and came to the conclusion that it might be better for all involved to drop support for Zun for the 2023.1 release until it can be updated to support the necessary dependencies. More specific details at the end of the message. With this in mind, the plan is to start this process in a week from now, unless a better solution is presented. There is still some time for discussion, please don't hesitate! There are two paths forward after this: If Zun can be updated to support the dependencies during the maintenance timeframe of 2023.1, then support would be restored. If no support is available at the time Caracal releases, however, it will be considered for removal in kolla and kolla-ansible. What this means: Kolla-ansible operators currently running Zun should hold off migration to 2023.1. Any development help would of course be appreciated. Please coordinate with the Zun project to help with development. Details on the dependency problems: * Currently, Zun has an external dependency on Docker 20.x, and uses a feature that has been removed in later versions of Docker. * This Docker feature also relies on an older version of etcd (3.3) * The mitigation [2] is to pin to the old versions of Docker and etcd, but this is not possible with Debian Bookworm hosts. Why this is a problem in Kolla-ansible (and Kolla): * This can't easily be solved by vendoring in an older version of etcd: the migration path it introduces to operators would incur a significant amount of risk. * Host level involvement is also particularly painful, the host version of Docker needs to connect to the etcd service running in the containers. * Even though the containers can be kept buildable and the configuration and inventory settings could be maintained, a working setup is not testable in CI. Thanks very much for everyone's patience, contributions and time! [1]: https://meetings.opendev.org/meetings/kolla/2023/kolla.2023-09-13-13.00.log… [2]: https://bugs.launchpad.net/zun/+bug/2007142

2 2

[all][tc][ptls] Eventlet: Python 3.12, and other concerning discoveries
by Jay Faulkner 20 Dec '23

20 Dec '23

Hi all, For those who aren't aware, eventlet (https://github.com/eventlet/eventlet) is a library that almost every OpenStack project depends on and uses heavily. It's the primary method in most OpenStack project for achieving any form of parallelism via greenthreading. A couple of weeks ago it was brought to my attention, in the #openstack-tc IRC channel, that the eventlet library was broken on python 3.12 due to changes to the SSL stdlib in python 3.12. I enlisted a G-Research Open Source Software (GR-OSS) developer from outside the OpenStack community, Itamar, to help dig into what would be required to get python 3.12 support for eventlet, and asked him to do an overall health check on the project. The results were extremely concerning, and are why I'm emailing the list today. First of all, the original concern -- python 3.12 support -- is being worked on right now; there is a PR up to migrate the SSL module here https://github.com/eventlet/eventlet/pull/817 and a PR up here: https://github.com/eventlet/eventlet/pull/820 to begin migrating from nose3, a deprecated testing framework used by eventlet which does not support python 3.12. It's worth noting that in the past (2020) https://github.com/eventlet/eventlet/issues/638 there was resistance to removing nose for testing due to at-the-time-desired python2.7 support. Even if they are able to be merged, these PRs only scratch the surface of the total effort needed to get eventlet working on python 3.12. Furthermore, activity on the eventlet project is extremely slim right now. There have been no commits to the primary branch in nearly eight months and it's been a single maintainer project for an extremely long time. There are also indications there may be external factors impacting the maintainers' availability. This lack of redundancy on a core library that we have no clear migration path off of is concerning as well. As a large community that uses eventlet; I hope we can improve it. Now we get to the cherry on top -- code quality and existing bugs. When Itamar began evaluating the overall state of the project, he found that approximately 80% of the unit tests were passing. Some of the failing unit tests were valid failures. In fact, one of the failure cases -- around locks not being properly greened on upgrade -- explains an Ironic database migrations unit test failure which we spent several weeks off-and-on debugging last cycle. Along these lines; although eventlet has CI configured; those tests have never passed: https://github.com/eventlet/eventlet/actions/workflows/test.yaml?query=is%3… . I don't have a specific proposal for a solution. I do know that right now, OpenStack services are built "all-in" on eventlet, with no obvious or easy migration path off. At least in the short term, keeping eventlet running well is critical to keeping OpenStack running well. I'm sending this email to raise an alarm about this situation, and to hope that community members can work together to solve the problem. Thanks for reading and your help, Jay Faulkner OpenStack TC Chair

8 15

[zaqar][tc] master and 2023.2 unit tests are broken
by Takashi Kajinami 19 Dec '23

19 Dec '23

Hello, Today I happened to notice that unit tests in zaqar is broken because some tests requires base test class from oslo.db which was removed in 13.0.0 release[1]. The 13.0.0 release of oslo.db was made during 2023.2 cucle so these tests are also broken in stable/2023.2 branch. In addition, py311 job was made voting in master but is broken and we probably need to fix bindep.txt so that it works in Debian 12. I'm sending this mail so that people maintaining the project is aware of the situation. I'll add [tc] tag because if the CI is not fixed then zaqar may become a new candidate for inactive projects. Thank you, Takashi [1] https://review.opendev.org/c/openstack/oslo.db/+/874239

2 1

Cyborg nova reports mdev-capable resource is not available
by Karl Kloppenborg 08 Dec '23

08 Dec '23

Hi Cyborg Team! Karl from Helm Team. When creating a VM with the correct flavor, the mdev gets created by cyborg agent and I can see it in the nodedev-list --cap mdev. However Nova then fails with: nova.virt.libvirt.driver [<removed>- - default default] Searching for available mdevs... _get_existing_mdevs_not_assigned /var/lib/openstack/lib/python3.10/site-packages/nova/virt/libvirt/driver.py :8357 2023-09-21 14:34:47.808 1901814 INFO nova.virt.libvirt.driver [<removed> - - default default] Available mdevs at: set(). 2023-09-21 14:34:47.809 1901814 DEBUG nova.virt.libvirt.driver [<removed> - - default default] No available mdevs where found. Creating an new one... _allocate_mdevs /var/lib/openstack/lib/python3.10/site-packages/nova/virt/libvirt/driv er.py:8496 2023-09-21 14:34:47.809 1901814 DEBUG nova.virt.libvirt.driver [<removed> - - default default] Attempting to create new mdev... _create_new_mediated_device /var/lib/openstack/lib/python3.10/site-packages/nova/virt/libvirt/driver.py:8385 2023-09-21 14:34:48.455 1901814 INFO nova.virt.libvirt.driver [<removed> - - default default] Failed to create mdev. No free space found among the following devices: ['pci_0000_4b_03_1', … <truncated list>]. 2023-09-21 14:34:48.456 1901814 ERROR nova.compute.manager [<removed> - - default default] [instance: 2026e2a2-b17a-43ab-adcb-62a907f58b51] Instance failed to spawn: nova.exception.ComputeResourcesUnavailable: Insufficient compute resources: mdev-capable resource is not available. Once this happened, ARQ removes the mdev and cleans up. I’ve got Cyborg 2023.2 running and have a device profile like so: karl@Karls-Air ~ % openstack accelerator device profile show e2b07e11-fe69-4f33-83fc-0f9e38adb7ae +-------------+---------------------------------------------------------------------------+ | Field | Value | +-------------+---------------------------------------------------------------------------+ | created_at | 2023-09-21 13:30:05+00:00 | | updated_at | None | | uuid | e2b07e11-fe69-4f33-83fc-0f9e38adb7ae | | name | VGPU_A40-Q48 | | groups | [{'resources:VGPU': '1', 'trait:CUSTOM_NVIDIA_2235_A40_48Q': 'required'}] | | description | None | +-------------+---------------------------------------------------------------------------+ karl@Karls-Air ~ % I can see the allocation candidate: karl@Karls-Air ~ % openstack allocation candidate list --resource VGPU=1 | grep A40 | 41 | VGPU=1 | 229bf15f-5689-3d2c-b37b-5c8439ea6a71 | VGPU=0/1 | OWNER_CYBORG,CUSTOM_NVIDIA_2235_A40_48Q | karl@Karls-Air ~ % Am I missing something critical here? Because I cannot seem to figure this out… have I got a PCI address wrong, or something? Any help from the Cyborg or Nova teams would be fantastic. Thanks, Karl.

4 9

[openstack-ansible] overriding host vars for compute_hosts
by pi3.14＠tuta.io 06 Dec '23

06 Dec '23

Hey folks, Unfortunately, I am not able to specify host specific variables for the compute hosts. I am trying to specify the `nova_user_ssl_*` variables for each compute node. When these variables are specified in `user_variables.yml` and then specified in `openstack_user_config.yml`, they don't actually end up overriding the variables. But they are recognised correctly in `openstack_inventory.json`. When I remove the variables from `user_variables.yml`, everything works correctly. Is it not possible to specify general variables in `user_variables.yml` and then overwrite them in `openstack_user_config.yml`? Example for `openstack_user_config.yml` (same Problem when using `host_vars` instead of `container_vars`): ``` compute_hosts: compute1: ip: 192.168.1.2 container_vars: nova_user_ssl_cert: "/path/to/cert1" nova_user_ssl_key: "/path/to/key1" nova_user_ssl_ca_cert: "/path/to/ca1" ``` thanks Roby

2 1

[cinder] 2024.1 (Caracal) R-17 virtual mid cycle (6 Dec, 2023)
by Rajat Dhasmana 06 Dec '23

06 Dec '23

Hello Argonauts, We will be conducting the first midcycle of 2024.1 (Caracal) on 6th December, 2023. Following are the details: Date: 6th December, 2023 Time: 1400-1600 UTC Meeting link: https://meet.google.com/auq-jiem-zxy Etherpad: https://etherpad.opendev.org/p/cinder-caracal-midcycles We have 2 weeks so please add your topics timely. Thanks Rajat Dhasmana

1 1

[cinder][nova][glance][ironic] restricting image metadata values to 255 chars
by Jorge Merlino 05 Dec '23

05 Dec '23

Hello stackers, It has come to our attention recently that nova, cinder, and glance treat image properties differently. This came up in a discussion on a patch to extend the allowable length of image metadata values in cinder [0]. Briefly: - glance allows image property values to be unrestricted in the schema, but effectively limited to 65535 bytes by the database. - cinder restricts the length of an image property value to 255 chars (by request schema) if you try to update the volume_image_properties on a volume, but if you create a volume from an image, cinder happily copies over all the image properties from glance, giving them an effective limit of 65535 bytes. - nova mostly truncates image properties values to 255 chars (it's a bit more complicated, but we can consider that to be nova's behavior for the purposes of this discussion). (We're not sure what ironic does, but they're in the subject line as a consumer of images and volumes.) Note that we're talking about metadata *values* here; the metadata keys are already restricted by charset to basically US-ASCII and 255 length. Values are allowed to be unicode. The point of this email is that it would be good for the services to be consistent about image metadata values. Thus, we propose that all services should restrict image metadata values to 255 unicode characters. (Which is sort of happening already, but not consistently.) The further point of this email is to find out what the impact of this restriction would be on current users. So please respond if you have a use case that would be affected by this change, and could not be addressed by, for example, instead of having one enormous value for a single key, the metadata was broken into a number of key/value pairs that would satisfy the 255 char restriction. Thanks for your attention to this! [0] https://review.opendev.org/c/openstack/cinder/+/868485

2 2

A few questions about OpenStack projects archival strategy
by arabat50＠gmail.com 04 Dec '23

04 Dec '23

Dear OpenStack team, I hope this email finds you well. I am Ali Arabat, a PhD student in Software Engineering at École de Technologie Supérieure in Montreal, University of Quebec, Canada, conducting research studies on the evolution of multi-component systems, among which OpenStack plays a significant role. Through this email, we would like to ask a few questions related to our study on OpenStack. Our study revealed a significant increase in the number of cross-component changes over time. However, Such cross-component changes have decreased after archiving a large number of projects beyond 500. Therefore, we would like to have clarification and answers to a few questions as follows: What are the main factors in archiving OpenStack projects? Are there any policies established by OpenStack to archive such projects i.e., end of life? Who generally decides to abandon a given project? We highly value your input and we genuinely appreciate your time. Thank you for considering our request, and we look forward to your valuable input. Best regards, --------------------------------------- Ali Arabat Ph.D. student in Software Engineering Department of Software Engineering and IT École de Technologie Supérieure (ÉTS) | University of Quebec 1100, rue Notre-Dame Ouest, Montréal, QC H3C 1K3, Canada

2 1

[ironic] multi-conductor architecture, standalone and DHCP
by Dmitry Tantsur 01 Dec '23

01 Dec '23

Hi folks, This is detailed context on the discussions we've had recently about multi-conductor in metal3. I'm sending it to potentially interested people and cc'ing the openstack ML. The metal3 side is https://github.com/dtantsur/ironic-operator/issues/3 (although that covers a few more different issues). In metal3 we have Ironic without Neutron (or really any other OpenStack components). So far, we have only supported a single Ironic instance in a cluster, but we want to move towards a multi-conductor setup where every Kubernetes control plane node will host an Ironic (so, 3 Ironics in a normal deployment). The biggest problem stems from DHCP for the provisioning network. With Neutron, Ironic can configure it to direct a node to boot from a specific iPXE server. With Metal3, we have a static DHCP configuration. If we keep it the way it is, it will always direct a node to boot from the iPXE server on the same machine as dnsmasq (regardless of how many dnsmasq replicas we run). This has to be changed. I see several options, none of which are straightforward. (1) iPXE-level fix: boot config API This is what https://bugs.launchpad.net/ironic/+bug/2044561 proposes. If we stop using static iPXE scripts and start going through the Ironic API instead, Ironic can reach out to the correct conductor internally, always serving the right script. (2) dnsmasq DHCP implementation Ironic now has a non-neutron managed DHCP implementation through DHCP host and options files: https://opendev.org/openstack/ironic/src/branch/master/ironic/dhcp/dnsmasq.…. This could be a half of the solution. Since Metal3 does not support new node auto-discovery, we can run 3 copies of dnsmasq with dhcp-ignore=!known and rely on this DHCP provider to configure only 1 dnsmasq for each node. The other 2 will ignore the requests. The problem is in the DHCP range. As far as I understand, the 3 dnsmasqs must be configured with disjoint ranges to prevent leases from clashing. This seems non-trivial with Kubernetes which expects to create identical pods. Adam has suggested using annotations from inside ironic-operator, which could work if we find a way to read them inside the pod after it has started (which seems to pose a chicken-and-egg problem). Other workarounds are possible if we go down this path. E.g. ironic-operator could split the DHCP range and creating a mapping hostIP:subrange,hostIP2:subrange2 and pass it to pods. Then each pod will pick its subrange based on its IP address. Huge downside: every time a Kubernetes control plane node is replaced or changes its IP, the whole Ironic installation will get restarted. (2.1) Pre-created leases Another Metal3 contributor has suggested requiring an IPAM implementation (metal3 actually has one) that will create leases centrally, allowing us to use the same DHCP range. The only problem: how to pass them to dnsmasq in runtime without restarting all pods? We could probably have a small sidecar container next to dnsmasq that watches the Kubernetes API for new leases in IPAM. Or... (2.2) Dynamic DHCP and Kea The same contributor is working on Kea support for their deployment: https://hackmd.io/@7RNIJtmvSIeNpSOMTW9oqA/SkWhyr7Hp. There seem to be limitations in the free version, which caused them to write a tiny downstream plugin: https://gitlab.com/Orange-OpenSource/kanod/kanod-kea/-/blob/controlled-leas…. Changing the DHCP server is a dramatic step, I'm not sure if our small community can afford that. It may also pose problems for downstreams like OpenShift, requiring Metal3 to support both servers in the long run. Any thoughts are welcome. Dmitry

3 3

[neutron] Neutron drivers meeting
by Brian Haley 01 Dec '23

01 Dec '23

Hello Neutrinos, Please remember tomorrow (Friday) we have a Neutron drivers meeting. This is the agenda: https://wiki.openstack.org/wiki/Meetings/NeutronDrivers There is currently one topic: * [RFE] Use call_monitor_timeout of oslo.messaging RPCClient instead of custom backoff mechanism and hardcoded timeouts https://bugs.launchpad.net/neutron/+bug/2045058 Thanks, -Brian

1 0

[OpenInfra Live]: PTG Recap, Dec 7 at 15 UTC
by Kristin Barrientos 30 Nov '23

30 Nov '23

Hi everyone, Thanks for watching this week's episode of OpenInfra Live, featuring Zvonko Kaiser from NVIDIA. In case you missed it, you can find it here: https://www.youtube.com/live/urHjDdtuIa4?feature=shared Tune in next week for another OpenInfra Live, brought to you by the OpenStack and StarlingX community! Episode: Join us for a Project Teams Gathering (PTG) recap! Project leaders from Ironic, Technical Committee, Nova, Sunbeam, Manila, StarlingX and Neutron will provide recaps from discussions held during the PTG. Date and time: Thursday, Dec 7, 15 UTC YouTube: https://www.youtube.com/live/thidlQGX29M?feature=shared LinkedIn: https://www.linkedin.com/events/7132754094531747840/comments/ WeChat: The recording will be posted on OpenInfra Foundation WeChat after the live stream Speakers: Kendall Nelson, Jay Faulkner, Sylvain Bauza, James Page, Carlos Eduardo, Steve Geary and Brian Haley Thanks, Kristin Barrientos Marketing Coordinator OpenInfra Foundation

1 0

"openstack usage list" doesn't return information about all projects
by Michel Jouvin 30 Nov '23

30 Nov '23

Hi, We are using "openstack usage list" to follow resource usage by projects at our cloud. We recently discovered that ~1/3 of the resource consumed (mostly "big projects") are missing from the JSON/CSV produced where the Admin Overview menu in the Horizon dashboard displays everything. Is there a trivial reason for this? We tried both with a pretty old openstackclient (5.8) and the last one after applying the patch in https://github.com/openstack/python-openstackclient/commit/30462f57d48162d1… to fix the problem with date conversion. The result is the same. Thank you in advance for any help. Cheers, Michel

5 15

openstack Vm shutoff by itself
by AJ_ sunny 30 Nov '23

30 Nov '23

Hi team, I am using openstack kolla ansible on wallaby version and currently I am facing issue with virtual machine, vm is shutoff by itself and and from log it seems libvirt lifecycle stop event triggering again and again Logs:- 2023-10-16 08:48:10.971 7 WARNING nova.compute.manager [req-c7b731db-2b61-400e-917f-8645c9984696 f226d81a45dd46488fb2e19515 848 316d215042914de190f5f9e1c8466bf0 default default] [instance: 4b04d3f1-1fbd-4b63-b693-a0ef316ecff3] Received unexpected - vent network-vif-plugged-f191f6c8-dff5-4c1b-94b3-8d91aa6ff5ac for instance with vm_state active and task_state None. 2023-10-21 22:42:44.589 7 INFO nova.compute.manager [-] [instance: 4b04d3f1-1fbd-4b63-b693-a0ef316ecff3] VM Stopped (Lifecyc Event) 2023-10-21 22:42:44.683 7 INFO nova.compute.manager [req-1d99b87b-7ff7-462d-ab18-fbdec6bda71d -] [instance: 4b04d3f1- fbd-4b63-b693-a0ef316ecff3] During _sync_instance_power_state the DB power_state (1) does not match the vm_power_state from ti e hypervisor (4). Updating power_state in the DB to match the hypervisor. 2023-10-21 22:42:44.811 7 WARNING nova.compute.manager [req-1d99b87b-7ff7-462d-ab18-fbdec6bda71d ----] [instance: 4b04d3f 1-1fbd-4b63-b693-a0ef316ecff3] Instance shutdown by itself. Calling the stop API. Current vm_state: active, current task_state : None, original DB power_state: 1, current VM power_state: 4 2023-10-21 22:42:44.977 7 INFO nova.compute.manager [req-1d99b87b-7ff7-462d-ab18-fbdec6bda71d -] [instance: 4b04d3f1-1 fbd-4b63-b693-a0ef316ecff3] Instance is already powered off in the hypervisor when stop is called. Thanks & Regards Arihant Jain +91 8299719369

5 15

How do I query the distribution versions of Nova and Neutron for openstack that have been configured？
by qq597399721＠163.com 30 Nov '23

30 Nov '23

How do I query the distribution versions of Nova and Neutron for openstack that have been configured？ I have tried using the "nova -- version" and "neutron -- version" commands, but the results of the commands show their CLI version, which is not their actual release version. I hope someone can help me find the correct command or path, thank you!

2 1

Failed to allocate the network(s), not rescheduling].
by kjme001＠gmail.com 30 Nov '23

30 Nov '23

hello, when it tries to create instances it gets an error message: Failed to allocate the network(s), not rescheduling]. as it restarts the openstack node it is already possible to create an instance without this error. What could be the reason, does anyone have an idea? Maybe someone will suggest something Greetings

2 2

[openstack][octavia] amphora cannot live migrate
by Nguyễn Hữu Khôi 30 Nov '23

30 Nov '23

Hello everyone. I deploy amphora instances using shared storage but when I do live migrate it to another host, I encounter this error. *Error: *Failed to live migrate instance to host "AUTO_SCHEDULE". Details <https://cloudhn.fpt.net/admin/instances/#message_details> compute12 is not on shared storage: Shared storage live-migration requires either shared storage or boot-from-volume with no local disks Woud we have a solution for this? Thank you. Nguyen Huu Khoi

3 11

Seeking Advice on Optimal Linux Distribution for OpenStack Installation
by Sait, Abubakur Abdulgader S 30 Nov '23

30 Nov '23

Hello All, I'm currently pursuing my PhD and my research involves utilizing OpenStack. I've encountered some challenges while attempting to install it on Ubuntu 22.04. I'd greatly appreciate it if you could recommend the most reliable Linux Distribution (along with the version) that has worked well for you. Additionally, any up-to-date tutorial suggestions for installing OpenStack would be incredibly helpful, as many of the resources I've come across seem to be outdated. Thanks for your assistance! Best regards, Bakursait

2 1

Cinder Bug Report 2023-11-29
by Jon Bernard 29 Nov '23

29 Nov '23

Hello Argonauts, Cinder Bug Meeting Etherpad <https://etherpad.opendev.org/p/cinder-bug-squad-meeting> Undecided - Virtual machine cannot unload data disk <https://bugs.launchpad.net/cinder/+bug/2044313> - Status: New Thanks, -- Jon

1 0

[neutron] Review day(s)
by Brian Haley 29 Nov '23

29 Nov '23

Hi Neutrinos, In our PTG and past weeks' meetings I've brought-up setting aside a day or two for us to prioritize merging some of our in-flight reviews, which currently number ~150 [0][1][2] just in the neutron* repositories. I'm hoping it will both help with our bug count as well as make neutron more stable, since a lot are fixing issues and/or adding new tests. I'm proposing we do this over the next two weeks if possible, I've made a doodle to gather responses at [3], that way we can try and coordinate reviewing and the inevitable re-factor time that comes along with merging code. Think of it as an early Christmas present :) Thanks for participating! -Brian, Neutron PTL [0] https://review.opendev.org/q/project:openstack%252Fneutron+branch:master+st… [1] https://review.opendev.org/q/project:openstack/neutron-lib+branch:master+st… [2] https://review.opendev.org/q/project:openstack/neutron-tempest-plugin+branc… [3] https://doodle.com/meeting/participate/id/dN8opRDa

1 1

[magnum] Weekly Meeting Cancelled - 29 Nov 2023
by Jake Yip 29 Nov '23

29 Nov '23

Hi all, Apologies for cancelling this week's meeting due to a clash. If you have a discussion topic, please add them to the agenda, and we will get to them next week. Regard, Jake

1 0

A questionnaire for unmaintained package users
by smoncaff＠gmail.com 29 Nov '23

29 Nov '23

1 0

Re: [Heat][requirements] Possible feature backport to stable branches
by Martin Magr 28 Nov '23

28 Nov '23

On Mon, Nov 13, 2023 at 4:50 PM Martin Magr <mmagr(a)redhat.com> wrote: > Adding more folks from requirements core to CC. > And also correct mailing list. I'm sorry for the noise. > > On Mon, Nov 13, 2023 at 4:32 PM Martin Magr <mmagr(a)redhat.com> wrote: > >> Greetings, >> >> we have introduced new feature to Aodh [1][2] (which introduced new >> dependency [3]) and Heat [4] recently. >> The changes to both projects are relatively small, it just added >> new respective resources and did not change >> existing logic. Also the new dependency is providing functionality to >> those new resources. >> >> We are building our product on stable/antelope and so ideal state for us >> would be to have those changes >> included in this branch also on upstream side. I'm aware that Heat >> follows stable branch policy and it's written >> in requirements documentation, that new dependency cannot be added in >> stable ranches, but before we are >> going to start building workarounds on this I'd like to ask if there is >> any chance of getting exception >> (given the fact that changes are pretty safe) on backporting the Heat >> patch down to stable/antelope >> and(or at least) including the new dependency in stable/antelope. >> >> Thanks for answer in advance, >> Martin >> >> [1] https://review.opendev.org/c/openstack/aodh/+/890529 >> [2] https://review.opendev.org/c/openstack/python-aodhclient/+/893319 >> [3] https://review.opendev.org/c/openstack/requirements/+/891291 >> [4] https://review.opendev.org/c/openstack/heat/+/893673 >> >> -- >> Martin Mágr >> Principal Software Engineer - Service Telemetry Framework >> Red Hat Czech >> > > -- Martin

5 6

[largescale-sig] Next meeting: Nov. 29, at 16UTC
by Kristin Barrientos 27 Nov '23

27 Nov '23

Hi everyone, The Large Scale SIG will be meeting this Wednesday in #openstack-operators on OFTC IRC, at 16UTC. Feel free to add topics to the agenda: https://etherpad.opendev.org/p/large-scale-sig-meeting Thanks, Kristin Barrientos Marketing Coordinator OpenInfra Foundation

1 0

[tc] Next Technical Committee Meeting Nov 28 1800 UTC
by Jay Faulkner 27 Nov '23

27 Nov '23

Hi all, The next weekly meeting of the Technical Committee is tomorrow, November 28, at 1800 UTC. It will be held in #openstack-tc in OFTC. Agenda - Roll call - Follow up on tracked action items: https://meetings.opendev.org/meetings/tc/2023/tc.2023-11-07-18.00.html - Rosmaita to propose amendment to unmaintained branch resolution allowing a single review group - Rosmaita to email mailing list about progress implementing unmaintained branch resolution - Gate health check - Leaderless projects (gmann) - Implementation of Unmaintained branch statuses - 2024.1 TC Tracker - https://etherpad.opendev.org/p/tc-2024.1-tracker - Upgrade issue between tooz and cinder - https://review.opendev.org/c/openstack/requirements/+/901131 - https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… - Declare projects inactive for lack of ...? - Working py311 unit tests - Support for sqla2 - Broken CI since 2023.2 release - Do we want/need to look at specific deliverables or only whole projects? - Open Discussion and Reviews Thanks, Jay Faulkner TC Chair

1 0

[all] North Dakota State University Student Opportunity
by Kendall Nelson 27 Nov '23

27 Nov '23

Hello All! I was contacted by a professor at North Dakota State University wondering if there were projects in OpenStack that students could work on over the course of the coming semester (Jan -May). The students will be nearing the end of their Bachelors degree (this course being one of the final requirements to graduate) so should be relatively knowledgeable and adept and picking up the technologies required if they aren't already versed in them. If you have an idea for a project that you would want to work with a group of students on, please let me know ASAP. I will get you connected with the professor. Project descriptions are due to the professor by December 1st so we only have a couple weeks to get them together. Anyone interested? -Kendall Nelson Senior Upstream Developer Advocate OpenInfra Foundation

1 2

Is "nova-manage db online_data_migrations" cumulative across versions?
by Michel Jouvin 27 Nov '23

27 Nov '23

Hi, We realized that after a Stein -> Train upgrade of our cloud, we forgot to run "nova-manage db online_data_migrations" and did the Train -> Ussuri upgrade. We have not seen any bad side effect but where wondering if the online_data_migrations was cumulative across versions and if running it under Ussuri fixed potential changes related to Stein -> Train upgrade that we forgot to fix right after the Train upgrade. If not, are there things to do to cleanup the situation? Cheers, Michel

2 1

[neutron] Bug Deputy Report Nov 20-26
by Oleg Bondarev 27 Nov '23

27 Nov '23

Hello Neutron Team, Easy week, as usual for me :P 1 bug is undecided, more info requested, others are on track or fixed. Critical ----- - https://bugs.launchpad.net/neutron/+bug/2043959 - Unit test jobs with neutron-lib master(3.9.0) failing - Fix released: https://review.opendev.org/c/openstack/neutron/+/901430 - Fixed by yatinkarel High ----- - https://bugs.launchpad.net/neutron/+bug/2044272 - Inconsistent IGMP configuration across drivers - In progress: https://review.opendev.org/c/openstack/neutron/+/901753 - assigned to lucasagomes Medium ----- - https://bugs.launchpad.net/neutron/+bug/2044171 - External shared networks may not be seen by other projects - Invalid, not relevant for master ( https://review.opendev.org/c/openstack/neutron-lib/+/901565 closed) - handled by libosvar - https://bugs.launchpad.net/neutron/+bug/2044484 - Enabling DHCP after setting the subnet's 'enable_dhcp' to false does not recreate the metadata namespace - In progress https://review.opendev.org/c/openstack/neutron/+/901812 - assigned to huanghailun Undecided ----- - https://bugs.launchpad.net/neutron/+bug/2044215 - dns: Keyword is replaced by project_name instead of project_id - Incomplete, more info requested Thanks, Oleg

1 0

[cinder][kolla][OpenstackAnsible] 2023.2 Bobcat cycle-trailing release deadline
by Elõd Illés 27 Nov '23

27 Nov '23

Hello teams with trailing projects, The 2023.2 Bobcat cycle-trailing release deadline is in ~1 months [1], and all projects following the cycle-trailing release model must release their 2023.2 Bobcat deliverables by December 7th, 2023. The following trailing projects haven't been released yet for 2023.2 Bobcat (aside the release candidates versions if exists). Cinder team's deliverables: - cinderlib OSA team's deliverables: - openstack-ansible-roles - openstack-ansible Kolla team's deliverables: - kayobe - kolla - kolla-ansible - ansible-collection-kolla This is just a friendly reminder to allow you to release these projects in time. Do not hesitate to ping us if you have any questions or concerns. [1] https://releases.openstack.org/caracal/schedule.html#c-cycle-trail Thanks, Előd irc: elodilles @ #openstack-release

1 1

[release][puppet] Can we retire old stable branches ?
by Takashi Kajinami 27 Nov '23

27 Nov '23

Hi release team, The puppet-ceph repository in Puppet OpenStack project currently follows the independent release model. This is because the module is more aligned with Ceph releases rather than OpenStack release. In the past, we created a stable branches per Ceph stable release and we currently have the following three stable branches. - stable/jewel - stable/mimic - stable/luminous However we no longer use these branches because the old stable branches (eg train) using these old releases are already EOL. So I'm wondering if someone from the release team can create -eol tag from the latest commits and remove these branches. Ideally we may be able to create a release from the latest commits, but I doublt we can merge any patches cleanly to these ancient branches so I tend to just add -eol tag instead, assuming no one may need releases for these old contents. Thank you, Takashi Kajinami

2 1

[all] [taskflow] asyncore is removed in Python 3.12
by Corey Bryant 27 Nov '23

27 Nov '23

Hi All, As noted in the following bug, taskflow uses asyncore, which is removed in Python 3.12. tasklfow needs to be switched to asyncio as a result. Does anyone happen to be working on this? https://bugs.launchpad.net/ubuntu/+source/python-taskflow/+bug/2024588 Thanks, Corey

5 7

[publiccloud-sig] Reminder - next meeting November 29th - 0800 UTC
by Felix Kronlage-Dammers 27 Nov '23

27 Nov '23

Hi everyone, since we’re switching to even weeks with the publiccloud-sig meeting (as also announced last week[1] in my summary), I’m happy to share the reminder for this weeks meeting :) This Wednesday the next meeting of the public cloud sig is going to happen. We meet on IRC in #openstack-operators at 0800 UTC. See also here for all other details: https://wiki.openstack.org/wiki/PublicCloudSIG felix [1] <url: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack…> -- Felix Kronlage-Dammers Product Owner IaaS & Operations Sovereign Cloud Stack Sovereign Cloud Stack — standardized, built and operated by many Ein Projekt der Open Source Business Alliance - Bundesverband für digitale Souveränität e.V. Tel.: +49-30-206539-205 | Matrix: @fkronlage:matrix.org | fkr(a)osb-alliance.com

1 0

[tripleo] stepping down from TripleO core
by Takashi Kajinami 27 Nov '23

27 Nov '23

Hello, As you might know or notice, I recently joined a different company, and my focus has been changed. I'm still involved in the OpenStack community but I may no longer be involved in TripleO. To clarify my status I've removed myself from TripleO core. # Please be aware that I'm removing myself from release liason, # someone from the core team might want to replace me, though # I don't think there may be multiple releases created in the future. I'd like to thank all members in the team (current members and ex-menmbers) for your great collaboration and work in the project. Thank you, Takashi Kajinami

3 2

[puppet] Proposing EOL of stable/wallaby and xena
by Takashi Kajinami 24 Nov '23

24 Nov '23

Hello, Some of you may know this but currently CentOS 8 Stream integration jobs in stable branches are all broken. I've looked into some of these failures but tasks randomly get stuck which eventually caused timeout and I've not yet found any potential cause. While we may be able to spend some time to dig into the problem, I'm wondering if anyone still requires wallaby and xena which support CentOS 8 Stream. Most of the backports submitted to these old branches were for TripleO,but I've seen some patches indicating TripleO wallaby is being retired so I doubt anyone still need keeping these branches open for backports. Please let me know if you have any concern about the proposed EOL and are interested in fixing these jobs. The stable/yoga branch has already passed the EM phase, so is another candidate for EOL but my intention is to keep it for now and re-consider its EOL early next year. We may have to make all CentOS 8 jobs non-voting, though. (We have CentOS 9 jobs in this branch). Thank you, Takashi Kajinami

2 2

[kolla] Upgrading from Train to Wallaby
by Albert Braden 24 Nov '23

24 Nov '23

Is it possible to upgrade kolla-ansible from Train to Wallaby in one step, or do we need to step through Ussuri and Victoria?

4 3

[keystone] temporary workaround for deprecated _member_ role?
by Christian Stelter 23 Nov '23

23 Nov '23

Hi! As someone who hasn't yet delved very deeply into keystone and the policies of the individual OpenStack services, I wondered whether setting member as the implied role for _member_ could pick up all those users who didn't manage to rotate their application credentials before switching to Zed Antelope. Is this a valid approach or could it cause problems? Kind regards, Christian

4 8

[ironic] Multiple vendor drivers being deprecated
by Julia Kreger 22 Nov '23

22 Nov '23

Greetings everyone, During Ironic's last PTG[0], we discussed the future path for drivers and the overall direction in the vendor ecosystem. The overall consensus was that DMTF Redfish is largely achieving its overall goal to standardize interfaces for interactions with Baseboard Management Controllers. Most of the vendors we collaborate with are in the process of determining their support and migration paths to Redfish. But as with any Open Source project with drivers, we also haven't heard from some vendors recently for a variety of reasons. It is with that in mind, we've made the decision to deprecate and eventually remove[1] some of the vendor drivers in cases where we've not heard from the maintainers and there is a forward path. * ibmc - We've not heard from the maintainer in some time due to geopolitical circumstances, but when we last heard from the developer/maintainer, they indicated work was actively underway to achieve redfish compliance in their employer's hardware. * xclarity - This driver was a driver which was designed to talk to the Lenovo xClarity Controller. This driver largely had no forward path, as we believe the underlying API support it used was also deprecated by Lenovo with the release of xClarity Controller v3. We have received numerous reports that the upstream redfish hardware type works as expected against Lenovo hardware, and at this point it just seems logical to move beyond this early driver. * iDRAC WSMAN driver flavors - The Dell team at a point long in the past decided Redfish was the future direction, and they created a set of "Redfish" interfaces in their driver, which replaced "WebServices for Management", better known as WSMAN interfaces which hail back to the days of iDRAC 6 (2008). At this point, given iDRAC7 (2012) firmware updates are available with Redfish support, we believe Redfish is generally suitable moving forward. The specific modules going away are the "-wsman" variant interfaces under the idrac hardware type. In case you are now wondering about the Fujitsu iRMC and HPE iLO drivers, we've heard from those teams recently and they too are focusing on Redfish as their forward path. The exact shape and form remains to be determined as each vendor has unique constraints and requirements to work with. As a result of this, their driver interfaces are not impacted by this decision by the community. If you have any questions or concerns, please feel free to reach out or join the Ironic team during our weekly meeting[2]. -Julia [0]: https://etherpad.opendev.org/p/ironic-ptg-october-2023 [1]: https://review.opendev.org/c/openstack/ironic/+/901501 [2]: https://wiki.openstack.org/wiki/Meetings/Ironic

1 0

Re: [SDK][CLI][API][OpenAPI] Status Update
by Artem Goncharov 22 Nov '23

22 Nov '23

Hi all, as promised here is status update of the OpenAPI work, and sorry in advance for another long message Short version ========== I have extended codegerator [1] to build OpenAPI specs from the sources of the services. All the exposed URLs with wherever possible query parameters/request/body are placed into the OpenAPI spec and whenever available generated information is extended with descriptions from rendered API-Ref html to keep life easier. This helped me to spot a variety of issues: unknown/undesired routes exposed, undocumented routes, documentation for unexposed routes, etc Sadly not all services are having all the required information to build the spec so the generator is created in the way that supports extension and overrides for every service. - Nova has route versions, request_body and response codes, but no information about response json - Glance is not bad, but not every request/response info is available directly on the controller. - Cinder is not having query parameters information, no response body schema - Keystone is not having much of useful information directly attached to the routes, but is having useful part schemas in the code - Neutron is very special - Octavia is having request and response info available (as wsme, but I have written wsme=>jsonschema convertor) Current state of schemas can be found under [2]. Schemas are still with gaps, but general structure can be seen [1] https://review.opendev.org/c/openstack/openstacksdk/+/882326 [2] https://github.com/gtema/openstack-openapi/tree/main/specs Long version ========= codegenerator is able to inspect source code of services in diverse ways (requires service to be "pip installed" into the env of codegenerator). Generally I am looking at the api router/mapper of the service and start traversing from there. This ensures we document all the routes and not what we think about. It helped me to figure out i.e. Cinder is exposing all routes with and without project_id (this was honestly new to me and would have helped to reduce insanity in project_id handling in the SDK code), Octavia exposes all routes with /v2, /v2/lbaas, /v2.0 and /v2.0/lbaas prefixes (altogether 4 times same operations). It is also possible to provide rendered html of the API-Ref and codegenerator will try to extract all methods descriptions and even try to get parameters description as long as it is possible (no magic, best effort approach). Nova: uses typical wsgi + routes. All exposed routes in all versions can be seen. There are validations attached to all routes what provides jsonschemas about query parameters and request body. There is some non standard mess with flavors actions (flavor delete/create/update are exposed as wsgi actions what under the hood exposes /flavors/{flavor_id}/action with corresponding json bodies. Validation for those fails, but that results in a wrong errors being returned to the user. I assume this is rather a bug and not a feature. Sadly there are no schemas describing api responses at all, so this needs to be provided separately/manually. Cinder: similar to Nova, except that also no query parameters information is available. Glance: Glance is sometimes having useful information attached to the routes by having serializer/deserializer classes available. This helped getting nearly all metadefs documented at no effort. Sadly there are also negative exceptions: image data operations are not having content-type information attached directly, so this requires a hack in the generator to build a correct spec for those operations. Still I was positively surprised with the amount of useful info I was able to get out of the source code without "hacks". Keystone: Flask is a tricky thing. Looking at all the routes exposed I got specs for all the resources in HEAD mode. Current API Ref mentions that, but they are not described at all. But that is not a big deal. A more severe problem is that the same controller is exposed for /<RESOURCE> and /<RESOURCE>/{resource_id} routes. This leads to the fact that user can send i.e. DELETE/PUT to /projects which will fail with 500 because project_id is a positional argument in the controller and not having default. But that heavily depends on the function signature and in my eyes this is a huge potential vulnerability and a quality gap (what would happen when somebody accidentally adds default value to that parameter?). This is the same for all routes exposed and not only relevant to projects. Sadly no extractable information about query parameters is available, but jsonschemas of resources are there (even for the /auth/tokens ). This requires manual mapping in the generator to know where schemas can be obtained from. Neutron: This one caused me a 4 day long headache. I have solved most of the issues, but there is still work to be done - inspecting the app requires neutron to be running (with a db created, all the schemas populated, config with all the plugins/extensions enabled, ....) Here the problem is to have actually all the plugins configured (some of them appears just don't like to work together with others) - some routes are exposed as wsgi app (the extensions app), but basic resources (networks, subnets, ports) are exposed as not inspectable pecan stuff (using dynamic _lookup to find route controller is just too complex for inspection) - neutron is having api definition which can be treated as a schema, but requires sophisticated logic (describing bulk requests is actually a no-brainer this way either) - still current biggest item is to have a max configuration so that codegenerator can look at every possible plugin Octavia: It is also using the Pecan framework with dynamic lookups which is a huge problem for the inspection, but the amount of those is rather an exception that allows me to traverse through "static" paths and supplement them with "exceptions". In contrast to all the other services Octavia is attaching full request and response definitions to the routes. Those are not jsonschema (wsme types), but jsonschema can be generated out of that with ~100 magic lines Heat: have not started, but the first look tells it is also using routes lib so it should be easy to get exposed routes. However it doesn't seem to have any usable api schemas in the code. Swift: I created the spec manually before I started working on generating specs from source. Luckily Swift is not having any/major api changes so it should not be a problem. With API-Ref info flowing into the spec we can get very usable specs automatically. Ideally all this information should be in docstrings (this is where initial descriptions are coming from and stay if api-ref match is not found). In the meanwhile I got requests about the topic from some other contributors which is why I feel myself confirmed I am not working on something nobody needs. As such we might need to restructure how and where the work is done and eventually get a few new projects to host codegenerator and openapi specs. Ideally I would like openapi specs to be belonging to services themselves, but I think getting them together separately is a better choice for the current phase until generation is finalized. Anyway, having a generator allows us to create jobs generating new specs for the services. With the generated specs I am already able to generate cli code (and sdk and ansible modules and etc) Next steps ======== - decide on where to host the generator (currently under the openstacksdk project) and where to store the specs - continue adding missing information to the already generated specs - decide on how to deal with sphinx-openapi. It is present and Stephen seems to be a maintainer of it, but even ignoring OpenStack API specialities it just produces not the output we used to have with current os-api-ref. So eventually a new project for rendering specs in the current style is necessary. Doesn't look like a complex task though - continue looking at other services (heat, magnum, etc) Feel free to ping me directly for collaboration and/or questions. Regards, Artem

2 1

[manila] Cancelling Nov 23rd weekly meeting
by Carlos Silva 22 Nov '23

22 Nov '23

Hello Zorillas! As a lot of people from our regular audience have holidays during this week, we are cancelling the Nov 23rd Manila weekly meeting. In the Nov 30th meeting, we would like to use most of the time to review backports, so please join us to help getting some fixes landing in the stable branches. Thanks and regards, carloss

1 0

[publiccloud-sig] Summary meeting CW 47
by Felix Kronlage-Dammers 22 Nov '23

22 Nov '23

Hi, in today’s publiccloud-sig meeting we’ve continued the topic of “flavor discoverability”, a topic that has been part of our discussion for a while. Before I dive into that, a quick heads-up: So far we’ve been meeting in the odd weeks. We’ll be switching that to the even weeks. This means, we’ll be meeting again next week (Nov. 29th). For today’s meeting: The minutes for the meeting are here: https://etherpad.opendev.org/p/publiccloud-sig-meeting Since this was a videocall (and as such no meetingbot logs), here a quick summary (thanks @joek-office). ======= 0. Discussion on flavor/image/volume "search engine" aka flavor-discovery - (gtema) The most informations that can be interessting for clients to search resources are at the respective services (nova/glance/cinder) in place - nova has traits but this is not searchable (possible extension - nova mentioned this as a tricky development), especially for clients - the traits has the ability to give out internal informations about the infrastructure, so cloud provider's possibly not happy with this solution - (gtema) problems with finding the right flavor/image in a openstack cloud - The flavor definitions are more or less a range in that nova scheduler will do its best to meet the criteria, no guarantee to get what you want @gtema pls update/delete this - The architecture of the instance is more defined by the image than from the flavor - thats because in the image are the appropriate sources for a cpu architecture - at the moment two possible ideas idea 1: extend the services with the appropriate features to meet the requirements cons: - long development pros: - no moderation/weighting of results possible idea 2: build up a new service with required metadata based on the .wellknown concept supported by sdk right now (or discoverable via service catalogue - that's not in scope of the service catalog, but the new "discovery" service may be added into the catalog) cons: pros: - the service can moderate/weight results to meet criterias from cloud provider AND client side - available for older OpenStack versions ========= For people interested in this subject, we (at least gtema and me) are going to continue this discussion tomorrow at 1400 CET here: https://conf.scs.koeln:8443/OIF-public-cloud-sig felix -- Felix Kronlage-Dammers Product Owner IaaS & Operations Sovereign Cloud Stack Sovereign Cloud Stack — standardized, built and operated by many Ein Projekt der Open Source Business Alliance - Bundesverband für digitale Souveränität e.V. Tel.: +49-30-206539-205 | Matrix: @fkronlage:matrix.org | fkr(a)osb-alliance.com

1 0

[kolla] Weekly meeting cancelled today (22nd Nov 2023)
by Michał Nasiadka 22 Nov '23

22 Nov '23

Hello, Due to key participants absence - the weekly Kolla meeting is cancelled today. Let’s meet next week Thanks, Michal

1 0

[cinder][backup] backup_max_operations configuration is not working
by Pureewat Kaewpoi 22 Nov '23

22 Nov '23

Hi communities I have set cinder backup with these configurations backup_max_operations = 4 backup_workers = 4 When I create 20 backups simultaneously According to logs It still creating backups at the same time. 2023-11-07 06:17:42.029 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:17:42.038 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:17:44.328 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:17:44.334 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:17:46.665 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:17:46.672 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:17:49.005 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:17:49.018 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:17:51.378 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:17:51.389 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:17:54.004 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:17:54.017 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:17:56.094 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:17:56.113 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:17:58.523 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:17:58.536 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:18:01.017 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:18:01.027 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:18:03.146 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:18:03.162 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:18:05.622 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:18:05.649 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:18:07.800 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:18:07.818 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:18:10.129 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:18:10.148 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:18:12.684 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:18:12.775 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:18:16.356 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:18:16.400 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:18:17.480 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:18:17.575 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:18:20.227 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:18:20.249 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:18:22.115 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:18:22.165 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:18:24.379 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:18:24.435 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:18:27.681 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:18:28.074 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:18:29.440 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:18:30.324 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:18:31.846 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:18:31.874 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) The expected result is creating 4 backups simultaneously Have anyone face this issue ? or Do I config a wrong configuration ? Thanks Pureewat Kaewpoi

3 3

[openstack][octavia] octavia scale
by Nguyễn Hữu Khôi 22 Nov '23

22 Nov '23

Hello everyone, I would like to know how can we scale octavia. For example, if octavia is full ram or cpu, how we can resize them. I just google and I cannot see docs about it. Thank you.

2 4

[openstack][magnum] cinder AZ for magmum
by Nguyễn Hữu Khôi 22 Nov '23

22 Nov '23

Hello guys. When I create k8s cluster with magnum, I can choose AZ for nodes but if I want to put my cluster to a specified cinder AZ. How can we do it? Thank you. Nguyen Huu Khoi

2 2

Nominate Nicolas Bock for charms-stable-maint
by Corey Bryant 21 Nov '23

21 Nov '23

I would like to nominate Nicolas Bock for charms-stable-maint. Nicolas has a solid history of contributions and reviews for various OpenStack projects, he is a previous PTL for the Designate project, and over the past 3+ years, he has been contributing to charm reviews [1]. I trust that Nicolas will continue to provide value and quality reviews for the OpenStack charms project. [1] https://review.opendev.org/q/reviewer:nicolas.bock%2540canonical.com+projec… Corey

5 5

[charms] Nominate Brian Haley for charms-stable-maint
by Corey Bryant 21 Nov '23

21 Nov '23

I would like to nominate Brian Haley for charms-stable-maint. Brian has a long history of contributions to OpenStack, is the currentl PTL for the Neutron project, and over the past year, Brian has also been contributing to charm reviews [1]. I have no doubt that Brian will continue providing quality reviews to the OpenStack charms project and will be a great asset in helping the project move forward. [1] https://review.opendev.org/q/reviewer:haleyb.dev%2540gmail.com+project:%255… Thanks, Corey

4 4

Isolating OpenStack Project
by open infra 21 Nov '23

21 Nov '23

Hi, Sorry for cross posting. I need to create a new project and expect to isolate all VMs and underline resources from other projects. I have a default 'admin' project with 20+ workers with a bunch of VMs which are already occupied. Is there a guide for this? My intention is to create a new project and all VMs are only scheduled on reserved workers where no other project is able to schedule workloads on those machines. My Openstack deployment is on top of StarlingX 7.0 We say isolation of complete projects, is it possible to count storage (ceph) as well? Regards, Danishka

2 1

RE: openstack-discuss - Openstack Kolla-ansible Nova Compute - Error | Urgent
by Asma Naz Shariq 21 Nov '23

21 Nov '23

Hi Berndbausch, Thankyou for your reply. I tried to resolve the compute node host in /etc/hosts file through (ip : computenode-virtual-machine). I can check that logs have been changed for nova-compute log on compute node as attached. I am unable to find the solution. Can you check this for me and guide me how to resolve these issues. I would appreciate if you schedule a session for it. T -----Original Message----- From: berndbausch(a)gmail.com <berndbausch(a)gmail.com> Sent: Monday, November 20, 2023 2:24 PM To: 'Asma Naz Shariq' <asma.naz(a)techavenue.biz>; openstack-discuss(a)lists.openstack.org Subject: RE: openstack-discuss -Nova Compute - Error | Urgent The compute node tries to register with Nova on the controller but can't communicate with it. The problem appears to be at RabbitMQ, which says it can't reach the controller. First things you should check: Unsynchronized clocks and incorrect configuration of name resolution on the compute node. -----Original Message----- From: Asma Naz Shariq <asma.naz(a)techavenue.biz> Sent: Monday, November 20, 2023 3:50 PM To: openstack-discuss(a)lists.openstack.org Subject: RE: openstack-discuss -Nova Compute - Error | Urgent Hi Openstack Community, I am deploying Openstack zed release Multinode cluster with 1x Controller and 1x compute and 1x storage. At the time of deployment, I encountered a below error for failed to register the compute service: TASK [nova-cell: Fail if nova-compute service failed to register] *********************************** fatal: [X.X.X.X]: FAILED! => {"changed": false, "msg": "The Nova compute service failed to register itself on the following hosts: computenode-virtual-machine"} fatal: [X.X.X.X]: FAILED! => {"changed": false, "msg": "The Nova compute service failed to register itself on the following hosts: computenode-virtual-machine"} The logs in nova-compute.log at compute node state the ERROR: 2023-11-20 11:54:32.008 7 ERROR oslo. messaging._drivers.impl_rabbit [None req-f8176572-75bf-4c27-a9be-e096cb43e2e3 - - - - - -] Connection failed: [Errno 113] EHOSTUNREACH (retrying in 31.0 seconds): OSError: [Errno 113] EHOSTUNREACH. Can anyone guide me how to resolve this issue? Thanks -----Original Message----- From: openstack-discuss-request(a)lists.openstack.org <openstack-discuss-request(a)lists.openstack.org> Sent: Monday, November 20, 2023 10:16 AM To: openstack-discuss(a)lists.openstack.org Subject: openstack-discuss Digest, Vol 61, Issue 70 Send openstack-discuss mailing list submissions to openstack-discuss(a)lists.openstack.org To subscribe or unsubscribe via email, send a message with subject or body 'help' to openstack-discuss-request(a)lists.openstack.org You can reach the person managing the list at openstack-discuss-owner(a)lists.openstack.org When replying, please edit your Subject line so it is more specific than "Re: Contents of openstack-discuss digest..." Today's Topics: 1. Re: about magnum capi for production (Nguyễn Hữu Khôi) ---------------------------------------------------------------------- Message: 1 Date: Mon, 20 Nov 2023 12:16:11 +0700 From: Nguyễn Hữu Khôi <nguyenhuukhoinw(a)gmail.com> Subject: Re: about magnum capi for production To: Satish Patel <satish.txt(a)gmail.com> Cc: OpenStack Discuss <openstack-discuss(a)lists.openstack.org> Message-ID: <CABAODRfcEA38QRLCpr_DtC5Dbxmr2x=G9nD8hENptAdh0Z0QXg(a)mail.gmail.com> Content-Type: multipart/alternative; boundary="000000000000068859060a8e96ec" Hello guys. This is a doc which is about how to make Magnum CAPI to work with Openstack deployed by Kolla Ansible.. https://github.com/ngyenhuukhoi/openstack-note/blob/main/Magnum%20Cluster%2… Feel free to ask. If I can help, I will. Nguyen Huu Khoi On Mon, Nov 20, 2023 at 12:09 PM Nguyễn Hữu Khôi <nguyenhuukhoinw(a)gmail.com> wrote: > Hello. > It uses prebuild Ubuntu to provision k8s cluster, > > On Mon, Nov 20, 2023 at 11:47 AM Satish Patel <satish.txt(a)gmail.com> > wrote: > >> Hi Folks, >> >> Tell me if I am wrong. CAPI is just kubernetes images to build k8s >> clusters instead using coreOS and install a bunch of stuff via heat. >> In documentation all I can see are imported images. Reference doc: >> https://vexxhost.github.io/magnum-cluster-api/user/getting-started/ >> >> Do I need to install anything else to make it work? >> >> On Fri, Nov 17, 2023 at 11:30 AM Mohammed Naser <mnaser(a)vexxhost.com> >> wrote: >> >>> Thanks! Happy to see that it is working smoothly 😊 >>> >>> >>> >>> Appreciate any docs, comments or issues. >>> >>> >>> >>> *From: *Nguyễn Hữu Khôi <nguyenhuukhoinw(a)gmail.com> >>> *Date: *Friday, November 17, 2023 at 8:57 AM >>> *To: *Mohammed Naser <mnaser(a)vexxhost.com> >>> *Cc: *OpenStack Discuss <openstack-discuss(a)lists.openstack.org> >>> *Subject: *Re: about magnum capi for production >>> >>> Hello guys. >>> >>> >>> >>> I have successfully implemented Magnum CAPI on my openstack system. >>> My deployment tool for Openstack is Kolla-Ansible. It worked >>> greatly, I am testing with create, delete and resize k8s cluster. It >>> is smooth when compared with Heat Driver. I will make some docs for new users to use it. >>> >>> >>> >>> Thank you vexxhost.com and contributors for very nice production. >>> >>> >>> >>> I will keep testing and reporting to help it be much better in the >>> future. >>> >>> >>> Nguyen Huu Khoi >>> >>> >>> >>> >>> >>> On Tue, Nov 14, 2023 at 5:50 AM Nguyễn Hữu Khôi < >>> nguyenhuukhoinw(a)gmail.com> wrote: >>> >>> Hello. >>> >>> I get it now; >>> >>> Thank you very much. >>> >>> Nguyen Huu Khoi >>> >>> >>> >>> >>> >>> On Mon, Nov 13, 2023 at 11:20 PM Mohammed Naser >>> <mnaser(a)vexxhost.com> >>> wrote: >>> >>> Hi there, >>> >>> >>> Nope – it is simply a different driver, so if you try to create >>> Fedora CoreOS clusters – it’ll use the old heat driver. Otherwise, >>> it will use the Cluster API if the image is marked as Ubuntu >>> >>> >>> >>> Thanks! >>> >>> Mohammed >>> >>> >>> >>> *From: *Nguyễn Hữu Khôi <nguyenhuukhoinw(a)gmail.com> >>> *Date: *Monday, November 13, 2023 at 1:58 AM >>> *To: *Mohammed Naser <mnaser(a)vexxhost.com> >>> *Cc: *OpenStack Discuss <openstack-discuss(a)lists.openstack.org> >>> *Subject: *Re: about magnum capi for production >>> >>> Hello Mohammed, >>> >>> >>> >>> I found that >>> >>> >>> >>> >>> https://vexxhost.github.io/magnum-cluster-api/user/getting-started/# >>> creating >>> >>> >>> >>> I would like to know that if we use cluster api, does it overwrite >>> heat when deploying new k8s cluster from horizon? >>> >>> >>> Nguyen Huu Khoi >>> >>> >>> >>> >>> >>> On Mon, Oct 30, 2023 at 7:49 PM Nguyễn Hữu Khôi < >>> nguyenhuukhoinw(a)gmail.com> wrote: >>> >>> Awesome, thank you so much! >>> >>> Nguyen Huu Khoi >>> >>> >>> >>> >>> >>> On Mon, Oct 30, 2023 at 7:31 PM Mohammed Naser <mnaser(a)vexxhost.com> >>> wrote: >>> >>> Sounds good, thank you. >>> >>> >>> >>> We also have a channel on the Kubernetes Slack that is relatively >>> active if you need help — it’s in the README :) >>> >>> >>> >>> Thanks! >>> >>> Mohammed >>> >>> >>> >>> Get Outlook for iOS <https://aka.ms/o0ukef> >>> ------------------------------ >>> >>> *From:* Nguyễn Hữu Khôi <nguyenhuukhoinw(a)gmail.com> >>> *Sent:* Monday, October 30, 2023 8:23:07 AM >>> *To:* Mohammed Naser <mnaser(a)vexxhost.com> >>> *Cc:* OpenStack Discuss <openstack-discuss(a)lists.openstack.org> >>> *Subject:* Re: about magnum capi for production >>> >>> >>> >>> Hello. >>> >>> Thank you very much. I am planning to use Magnum CAPI for my system. >>> I will feedback later. >>> >>> Nguyen Huu Khoi >>> >>> >>> >>> >>> >>> On Mon, Oct 30, 2023 at 7:03 PM Mohammed Naser <mnaser(a)vexxhost.com> >>> wrote: >>> >>> Hi there! >>> >>> >>> >>> It works out of the box since the goal is to have an API that is >>> compatible so that all the ecosystem tools such as Terraform, >>> Ansible, CLI, Horizon all continue to work just fine! >>> >>> >>> >>> We’ve even made some quality of life improvements to the Magnum >>> plugin like letting you download KUBECONFIG from it. >>> >>> >>> >>> Thanks >>> >>> Mohammed >>> >>> >>> >>> Get Outlook for iOS <https://aka.ms/o0ukef> >>> ------------------------------ >>> >>> *From:* Nguyễn Hữu Khôi <nguyenhuukhoinw(a)gmail.com> >>> *Sent:* Monday, October 30, 2023 1:09:10 AM >>> *To:* Mohammed Naser <mnaser(a)vexxhost.com> >>> *Cc:* OpenStack Discuss <openstack-discuss(a)lists.openstack.org> >>> *Subject:* Re: about magnum capi for production >>> >>> >>> >>> Hello, >>> >>> >>> >>> Thank you for your sharing, Could you tell me how we use it with >>> Horizon or we need custom Horizon to use Magnum CAPI. >>> >>> >>> >>> Nguyen Huu Khoi >>> >>> >>> >>> >>> >>> On Mon, Oct 30, 2023 at 11:10 AM Mohammed Naser >>> <mnaser(a)vexxhost.com> >>> wrote: >>> >>> Hi there. >>> >>> >>> >>> We deploy this driver in production in our public cloud as well as >>> many private clouds and as part of Atmosphere and we’ve been fully >>> maintaining it on our side here. >>> >>> >>> >>> Let me know if you have any questions >>> >>> >>> >>> Thanks >>> >>> Mohammed >>> >>> >>> >>> Get Outlook for iOS <https://aka.ms/o0ukef> >>> ------------------------------ >>> >>> *From:* Nguyễn Hữu Khôi <nguyenhuukhoinw(a)gmail.com> >>> *Sent:* Sunday, October 29, 2023 10:22:22 PM >>> *To:* OpenStack Discuss <openstack-discuss(a)lists.openstack.org> >>> *Subject:* about magnum capi for production >>> >>> >>> >>> Hello guys. >>> >>> Could you share with me some experience about using Magnum CAPI in >>> production? >>> >>> >>> >>> https://github.com/vexxhost/magnum-cluster-api >>> >>> >>> >>> Thank you so much. >>> >>> >>> Nguyen Huu Khoi >>> >>>

2 2

[SQLAlchemy 2.x][cloudkitty][ironic-inspector][masakari][murano][trove][sahara][senlin][vitrage][zaqar] No SQLA 2.x support in a few projects
by Thomas Goirand 20 Nov '23

20 Nov '23

Hi, As in Debian, a few packages need SQLAlchemy 2.x, I am working toward adding all SQLA 2.x patches as backport to Bobcat. So far, I managed to backport patches for Heat and Glance, and I'll work on Cinder and Manila next. However, a few packages are failing when using SQLAlchemy 2.x, as per the experimental pseudo-excuse page for sqlalchemy: https://qa.debian.org/excuses.php?experimental=1&package=sqlalchemy So, the list of packages is (in alphabetical order): - cloudkitty - ironic-inspector - masakari - murano - trove - sahara - senlin - vitrage - zaqar These projects need to urgently work toward supporting SQLAlchemy 2.x, both because SQLA 2 will be in distros, and because the OpenStack project requirements needs to move on and bump SQLA to 2.0. Is there anyone working on this? I'm especially interested having Cloudkitty fixed, since we use it in production, but I'm not sure I have the skills for it. Stephen, who wrote the patches for Cinder, Manila, Glance, Heat and Nova at least, doesn't have the time to work on these, he told me on IRC. Cheers, Thomas Goirand (zigo)

4 5

[neutron] Bug Deputy Report November 13-19
by Jakub Libosvar 20 Nov '23

20 Nov '23

Hi all, this is the bug report for Neutron for the week of November 13. There is nothing that requires immediate attention, one critical bug that is handled and about to merge the fix as of now. Critical ———— - Unit test jobs with neutron-lib master(3.9.0) failing https://bugs.launchpad.net/neutron/+bug/2043959 Assigned to: Yatin Fix proposed: https://review.opendev.org/c/openstack/neutron/+/901430 Medium ——— - Some init_router_port steps are unnecessary for new devices https://bugs.launchpad.net/neutron/+bug/2043558 Fix proposed: https://review.opendev.org/c/openstack/neutron/+/900553 Assigned to: Adam Oswick - list_ip_routes should only fetch needed devices https://bugs.launchpad.net/neutron/+bug/2043564 Fix proposed: https://review.opendev.org/c/openstack/neutron/+/900415 Assigned to: Adam Oswick - Creating subnet for exsiting external network is time-comsuming https://bugs.launchpad.net/neutron/+bug/2043761 Assigned to: Liu Fix proposed: https://review.opendev.org/c/openstack/neutron/+/901225 Duplicate ———— - [ovn] no need rpc notifier if no l2 agent https://bugs.launchpad.net/neutron/+bug/2043759 duplicate of https://bugs.launchpad.net/neutron/+bug/1992352

1 0

[publiccloud-sig] Reminder - next meeting November 22nd - 0800 UTC
by Felix Kronlage-Dammers 20 Nov '23

20 Nov '23

Hi everyone, this Wednesday the next meeting of the public cloud sig is going to happen. Since we do every other meeting as a videosession, we’ll be meeting at 0800 UTC here: https://conf.scs.koeln:8443/OIF-public-cloud-sig In parallel we will be on irc on #openstack-operators as well of course. Am very much looking forward to wednesday morning! :) felix -- Felix Kronlage-Dammers Product Owner IaaS & Operations Sovereign Cloud Stack Sovereign Cloud Stack — standardized, built and operated by many Ein Projekt der Open Source Business Alliance - Bundesverband für digitale Souveränität e.V. Tel.: +49-30-206539-205 | Matrix: @fkronlage:matrix.org | fkr(a)osb-alliance.com

1 0

RE: openstack-discuss -Nova Compute - Error | Urgent
by Asma Naz Shariq 20 Nov '23

20 Nov '23

Hi Openstack Community, I am deploying Openstack zed release Multinode cluster with 1x Controller and 1x compute and 1x storage. At the time of deployment, I encountered a below error for failed to register the compute service: TASK [nova-cell: Fail if nova-compute service failed to register] *********************************** fatal: [X.X.X.X]: FAILED! => {"changed": false, "msg": "The Nova compute service failed to register itself on the following hosts: computenode-virtual-machine"} fatal: [X.X.X.X]: FAILED! => {"changed": false, "msg": "The Nova compute service failed to register itself on the following hosts: computenode-virtual-machine"} The logs in nova-compute.log at compute node state the ERROR: 2023-11-20 11:54:32.008 7 ERROR oslo. messaging._drivers.impl_rabbit [None req-f8176572-75bf-4c27-a9be-e096cb43e2e3 - - - - - -] Connection failed: [Errno 113] EHOSTUNREACH (retrying in 31.0 seconds): OSError: [Errno 113] EHOSTUNREACH. Can anyone guide me how to resolve this issue? Thanks -----Original Message----- From: openstack-discuss-request(a)lists.openstack.org <openstack-discuss-request(a)lists.openstack.org> Sent: Monday, November 20, 2023 10:16 AM To: openstack-discuss(a)lists.openstack.org Subject: openstack-discuss Digest, Vol 61, Issue 70 Send openstack-discuss mailing list submissions to openstack-discuss(a)lists.openstack.org To subscribe or unsubscribe via email, send a message with subject or body 'help' to openstack-discuss-request(a)lists.openstack.org You can reach the person managing the list at openstack-discuss-owner(a)lists.openstack.org When replying, please edit your Subject line so it is more specific than "Re: Contents of openstack-discuss digest..." Today's Topics: 1. Re: about magnum capi for production (Nguyễn Hữu Khôi) ---------------------------------------------------------------------- Message: 1 Date: Mon, 20 Nov 2023 12:16:11 +0700 From: Nguyễn Hữu Khôi <nguyenhuukhoinw(a)gmail.com> Subject: Re: about magnum capi for production To: Satish Patel <satish.txt(a)gmail.com> Cc: OpenStack Discuss <openstack-discuss(a)lists.openstack.org> Message-ID: <CABAODRfcEA38QRLCpr_DtC5Dbxmr2x=G9nD8hENptAdh0Z0QXg(a)mail.gmail.com> Content-Type: multipart/alternative; boundary="000000000000068859060a8e96ec" Hello guys. This is a doc which is about how to make Magnum CAPI to work with Openstack deployed by Kolla Ansible.. https://github.com/ngyenhuukhoi/openstack-note/blob/main/Magnum%20Cluster%2… Feel free to ask. If I can help, I will. Nguyen Huu Khoi On Mon, Nov 20, 2023 at 12:09 PM Nguyễn Hữu Khôi <nguyenhuukhoinw(a)gmail.com> wrote: > Hello. > It uses prebuild Ubuntu to provision k8s cluster, > > On Mon, Nov 20, 2023 at 11:47 AM Satish Patel <satish.txt(a)gmail.com> > wrote: > >> Hi Folks, >> >> Tell me if I am wrong. CAPI is just kubernetes images to build k8s >> clusters instead using coreOS and install a bunch of stuff via heat. >> In documentation all I can see are imported images. Reference doc: >> https://vexxhost.github.io/magnum-cluster-api/user/getting-started/ >> >> Do I need to install anything else to make it work? >> >> On Fri, Nov 17, 2023 at 11:30 AM Mohammed Naser <mnaser(a)vexxhost.com> >> wrote: >> >>> Thanks! Happy to see that it is working smoothly 😊 >>> >>> >>> >>> Appreciate any docs, comments or issues. >>> >>> >>> >>> *From: *Nguyễn Hữu Khôi <nguyenhuukhoinw(a)gmail.com> >>> *Date: *Friday, November 17, 2023 at 8:57 AM >>> *To: *Mohammed Naser <mnaser(a)vexxhost.com> >>> *Cc: *OpenStack Discuss <openstack-discuss(a)lists.openstack.org> >>> *Subject: *Re: about magnum capi for production >>> >>> Hello guys. >>> >>> >>> >>> I have successfully implemented Magnum CAPI on my openstack system. >>> My deployment tool for Openstack is Kolla-Ansible. It worked >>> greatly, I am testing with create, delete and resize k8s cluster. It >>> is smooth when compared with Heat Driver. I will make some docs for new users to use it. >>> >>> >>> >>> Thank you vexxhost.com and contributors for very nice production. >>> >>> >>> >>> I will keep testing and reporting to help it be much better in the >>> future. >>> >>> >>> Nguyen Huu Khoi >>> >>> >>> >>> >>> >>> On Tue, Nov 14, 2023 at 5:50 AM Nguyễn Hữu Khôi < >>> nguyenhuukhoinw(a)gmail.com> wrote: >>> >>> Hello. >>> >>> I get it now; >>> >>> Thank you very much. >>> >>> Nguyen Huu Khoi >>> >>> >>> >>> >>> >>> On Mon, Nov 13, 2023 at 11:20 PM Mohammed Naser >>> <mnaser(a)vexxhost.com> >>> wrote: >>> >>> Hi there, >>> >>> >>> Nope – it is simply a different driver, so if you try to create >>> Fedora CoreOS clusters – it’ll use the old heat driver. Otherwise, >>> it will use the Cluster API if the image is marked as Ubuntu >>> >>> >>> >>> Thanks! >>> >>> Mohammed >>> >>> >>> >>> *From: *Nguyễn Hữu Khôi <nguyenhuukhoinw(a)gmail.com> >>> *Date: *Monday, November 13, 2023 at 1:58 AM >>> *To: *Mohammed Naser <mnaser(a)vexxhost.com> >>> *Cc: *OpenStack Discuss <openstack-discuss(a)lists.openstack.org> >>> *Subject: *Re: about magnum capi for production >>> >>> Hello Mohammed, >>> >>> >>> >>> I found that >>> >>> >>> >>> >>> https://vexxhost.github.io/magnum-cluster-api/user/getting-started/# >>> creating >>> >>> >>> >>> I would like to know that if we use cluster api, does it overwrite >>> heat when deploying new k8s cluster from horizon? >>> >>> >>> Nguyen Huu Khoi >>> >>> >>> >>> >>> >>> On Mon, Oct 30, 2023 at 7:49 PM Nguyễn Hữu Khôi < >>> nguyenhuukhoinw(a)gmail.com> wrote: >>> >>> Awesome, thank you so much! >>> >>> Nguyen Huu Khoi >>> >>> >>> >>> >>> >>> On Mon, Oct 30, 2023 at 7:31 PM Mohammed Naser <mnaser(a)vexxhost.com> >>> wrote: >>> >>> Sounds good, thank you. >>> >>> >>> >>> We also have a channel on the Kubernetes Slack that is relatively >>> active if you need help — it’s in the README :) >>> >>> >>> >>> Thanks! >>> >>> Mohammed >>> >>> >>> >>> Get Outlook for iOS <https://aka.ms/o0ukef> >>> ------------------------------ >>> >>> *From:* Nguyễn Hữu Khôi <nguyenhuukhoinw(a)gmail.com> >>> *Sent:* Monday, October 30, 2023 8:23:07 AM >>> *To:* Mohammed Naser <mnaser(a)vexxhost.com> >>> *Cc:* OpenStack Discuss <openstack-discuss(a)lists.openstack.org> >>> *Subject:* Re: about magnum capi for production >>> >>> >>> >>> Hello. >>> >>> Thank you very much. I am planning to use Magnum CAPI for my system. >>> I will feedback later. >>> >>> Nguyen Huu Khoi >>> >>> >>> >>> >>> >>> On Mon, Oct 30, 2023 at 7:03 PM Mohammed Naser <mnaser(a)vexxhost.com> >>> wrote: >>> >>> Hi there! >>> >>> >>> >>> It works out of the box since the goal is to have an API that is >>> compatible so that all the ecosystem tools such as Terraform, >>> Ansible, CLI, Horizon all continue to work just fine! >>> >>> >>> >>> We’ve even made some quality of life improvements to the Magnum >>> plugin like letting you download KUBECONFIG from it. >>> >>> >>> >>> Thanks >>> >>> Mohammed >>> >>> >>> >>> Get Outlook for iOS <https://aka.ms/o0ukef> >>> ------------------------------ >>> >>> *From:* Nguyễn Hữu Khôi <nguyenhuukhoinw(a)gmail.com> >>> *Sent:* Monday, October 30, 2023 1:09:10 AM >>> *To:* Mohammed Naser <mnaser(a)vexxhost.com> >>> *Cc:* OpenStack Discuss <openstack-discuss(a)lists.openstack.org> >>> *Subject:* Re: about magnum capi for production >>> >>> >>> >>> Hello, >>> >>> >>> >>> Thank you for your sharing, Could you tell me how we use it with >>> Horizon or we need custom Horizon to use Magnum CAPI. >>> >>> >>> >>> Nguyen Huu Khoi >>> >>> >>> >>> >>> >>> On Mon, Oct 30, 2023 at 11:10 AM Mohammed Naser >>> <mnaser(a)vexxhost.com> >>> wrote: >>> >>> Hi there. >>> >>> >>> >>> We deploy this driver in production in our public cloud as well as >>> many private clouds and as part of Atmosphere and we’ve been fully >>> maintaining it on our side here. >>> >>> >>> >>> Let me know if you have any questions >>> >>> >>> >>> Thanks >>> >>> Mohammed >>> >>> >>> >>> Get Outlook for iOS <https://aka.ms/o0ukef> >>> ------------------------------ >>> >>> *From:* Nguyễn Hữu Khôi <nguyenhuukhoinw(a)gmail.com> >>> *Sent:* Sunday, October 29, 2023 10:22:22 PM >>> *To:* OpenStack Discuss <openstack-discuss(a)lists.openstack.org> >>> *Subject:* about magnum capi for production >>> >>> >>> >>> Hello guys. >>> >>> Could you share with me some experience about using Magnum CAPI in >>> production? >>> >>> >>> >>> https://github.com/vexxhost/magnum-cluster-api >>> >>> >>> >>> Thank you so much. >>> >>> >>> Nguyen Huu Khoi >>> >>>

2 1

kolla-toolbox problem in multinode deploy
by Bursztynowski Dariusz 20 Nov '23

20 Nov '23

Hello, I'm facing a problem with kolla-toolbox while deploying zed-ubunu-jammy-aarch64 on a Raspberry Pi cluster using kolla-ansible. I 've built the images myself (on Raspbbery Pi) using kolla-build because images for Ubuntu-aarch64 were missing in standard repos (for Debian/aarch64, stuff installs from quay.io and works fine, but wanted to try out Ubuntu). The error can be seen at the very end of a fragment of the ansible log pasted below. As seen, images in my local repo are reachable (fluentd & kolla-toolbox restart succesfully). But in task "Initializing kolla-toolbox container" three hosts fail with the notification "Container xyz is restarting, wait until the container is running", while the fourth one gives "unable to upgrade to tcp, received 409". Interestingly, in a preceding task "Ensuring config directories have correct owner and permission" (the very first part of the log below), kolla-toolbox check is skipped. I can provide more data, create a bug report in appropriate place, etc. But please instruct me what and where. Best regards, Darek TASK [common : Ensuring config directories have correct owner and permission] ************************************************************************************************ ok: [ost64] => (item={'key': 'fluentd', 'value': {'container_name': 'fluentd', 'group': 'fluentd', 'enabled': True, 'image': '192.168.2.59:443/openstack.kolla/fluentd:zed-ubuntu-jammy-aarch64', 'environment': {'KOLLA_CONFIG_STRATEGY': 'COPY_ALWAYS'}, 'volumes': ['/etc/kolla/fluentd/:/var/lib/kolla/config_files/:ro', '/etc/localtime:/etc/localtime:ro', '/etc/timezone:/etc/timezone:ro', 'kolla_logs:/var/log/kolla/', 'fluentd_data:/var/lib/fluentd/data/'], 'dimensions': {}}}) ok: [ost61] => (item={'key': 'fluentd', 'value': {'container_name': 'fluentd', 'group': 'fluentd', 'enabled': True, 'image': '192.168.2.59:443/openstack.kolla/fluentd:zed-ubuntu-jammy-aarch64', 'environment': {'KOLLA_CONFIG_STRATEGY': 'COPY_ALWAYS'}, 'volumes': ['/etc/kolla/fluentd/:/var/lib/kolla/config_files/:ro', '/etc/localtime:/etc/localtime:ro', '/etc/timezone:/etc/timezone:ro', 'kolla_logs:/var/log/kolla/', 'fluentd_data:/var/lib/fluentd/data/'], 'dimensions': {}}}) skipping: [ost64] => (item={'key': 'kolla-toolbox', 'value': {'container_name': 'kolla_toolbox', 'group': 'kolla-toolbox', 'enabled': True, 'image': '192.168.2.59:443/openstack.kolla/kolla-toolbox:zed-ubuntu-jammy-aarch64', 'environment': {'ANSIBLE_NOCOLOR': '1', 'ANSIBLE_LIBRARY': '/usr/share/ansible'}, 'privileged': True, 'volumes': ['/etc/kolla/kolla-toolbox/:/var/lib/kolla/config_files/:ro', '/etc/localtime:/etc/localtime:ro', '/etc/timezone:/etc/timezone:ro', '/dev/:/dev/', '/run/:/run/:shared', 'kolla_logs:/var/log/kolla/'], 'dimensions': {}}}) skipping: [ost61] => (item={'key': 'kolla-toolbox', 'value': {'container_name': 'kolla_toolbox', 'group': 'kolla-toolbox', 'enabled': True, 'image': '192.168.2.59:443/openstack.kolla/kolla-toolbox:zed-ubuntu-jammy-aarch64', 'environment': {'ANSIBLE_NOCOLOR': '1', 'ANSIBLE_LIBRARY': '/usr/share/ansible'}, 'privileged': True, 'volumes': ['/etc/kolla/kolla-toolbox/:/var/lib/kolla/config_files/:ro', '/etc/localtime:/etc/localtime:ro', '/etc/timezone:/etc/timezone:ro', '/dev/:/dev/', '/run/:/run/:shared', 'kolla_logs:/var/log/kolla/'], 'dimensions': {}}}) ok: [ost62] => (item={'key': 'fluentd', 'value': {'container_name': 'fluentd', 'group': 'fluentd', 'enabled': True, 'image': '192.168.2.59:443/openstack.kolla/fluentd:zed-ubuntu-jammy-aarch64', 'environment': {'KOLLA_CONFIG_STRATEGY': 'COPY_ALWAYS'}, 'volumes': ['/etc/kolla/fluentd/:/var/lib/kolla/config_files/:ro', '/etc/localtime:/etc/localtime:ro', '/etc/timezone:/etc/timezone:ro', 'kolla_logs:/var/log/kolla/', 'fluentd_data:/var/lib/fluentd/data/'], 'dimensions': {}}}) skipping: [ost62] => (item={'key': 'kolla-toolbox', 'value': {'container_name': 'kolla_toolbox', 'group': 'kolla-toolbox', 'enabled': True, 'image': '192.168.2.59:443/openstack.kolla/kolla-toolbox:zed-ubuntu-jammy-aarch64', 'environment': {'ANSIBLE_NOCOLOR': '1', 'ANSIBLE_LIBRARY': '/usr/share/ansible'}, 'privileged': True, 'volumes': ['/etc/kolla/kolla-toolbox/:/var/lib/kolla/config_files/:ro', '/etc/localtime:/etc/localtime:ro', '/etc/timezone:/etc/timezone:ro', '/dev/:/dev/', '/run/:/run/:shared', 'kolla_logs:/var/log/kolla/'], 'dimensions': {}}}) ok: [ost63] => (item={'key': 'fluentd', 'value': {'container_name': 'fluentd', 'group': 'fluentd', 'enabled': True, 'image': '192.168.2.59:443/openstack.kolla/fluentd:zed-ubuntu-jammy-aarch64', 'environment': {'KOLLA_CONFIG_STRATEGY': 'COPY_ALWAYS'}, 'volumes': ['/etc/kolla/fluentd/:/var/lib/kolla/config_files/:ro', '/etc/localtime:/etc/localtime:ro', '/etc/timezone:/etc/timezone:ro', 'kolla_logs:/var/log/kolla/', 'fluentd_data:/var/lib/fluentd/data/'], 'dimensions': {}}}) skipping: [ost63] => (item={'key': 'kolla-toolbox', 'value': {'container_name': 'kolla_toolbox', 'group': 'kolla-toolbox', 'enabled': True, 'image': '192.168.2.59:443/openstack.kolla/kolla-toolbox:zed-ubuntu-jammy-aarch64', 'environment': {'ANSIBLE_NOCOLOR': '1', 'ANSIBLE_LIBRARY': '/usr/share/ansible'}, 'privileged': True, 'volumes': ['/etc/kolla/kolla-toolbox/:/var/lib/kolla/config_files/:ro', '/etc/localtime:/etc/localtime:ro', '/etc/timezone:/etc/timezone:ro', '/dev/:/dev/', '/run/:/run/:shared', 'kolla_logs:/var/log/kolla/'], 'dimensions': {}}}) ok: [ost64] => (item={'key': 'cron', 'value': {'container_name': 'cron', 'group': 'cron', 'enabled': True, 'image': '192.168.2.59:443/openstack.kolla/cron:zed-ubuntu-jammy-aarch64', 'environment': {'KOLLA_LOGROTATE_SCHEDULE': 'daily'}, 'volumes': ['/etc/kolla/cron/:/var/lib/kolla/config_files/:ro', '/etc/localtime:/etc/localtime:ro', '/etc/timezone:/etc/timezone:ro', 'kolla_logs:/var/log/kolla/'], 'dimensions': {}}}) ok: [ost61] => (item={'key': 'cron', 'value': {'container_name': 'cron', 'group': 'cron', 'enabled': True, 'image': '192.168.2.59:443/openstack.kolla/cron:zed-ubuntu-jammy-aarch64', 'environment': {'KOLLA_LOGROTATE_SCHEDULE': 'daily'}, 'volumes': ['/etc/kolla/cron/:/var/lib/kolla/config_files/:ro', '/etc/localtime:/etc/localtime:ro', '/etc/timezone:/etc/timezone:ro', 'kolla_logs:/var/log/kolla/'], 'dimensions': {}}}) ok: [ost62] => (item={'key': 'cron', 'value': {'container_name': 'cron', 'group': 'cron', 'enabled': True, 'image': '192.168.2.59:443/openstack.kolla/cron:zed-ubuntu-jammy-aarch64', 'environment': {'KOLLA_LOGROTATE_SCHEDULE': 'daily'}, 'volumes': ['/etc/kolla/cron/:/var/lib/kolla/config_files/:ro', '/etc/localtime:/etc/localtime:ro', '/etc/timezone:/etc/timezone:ro', 'kolla_logs:/var/log/kolla/'], 'dimensions': {}}}) ok: [ost63] => (item={'key': 'cron', 'value': {'container_name': 'cron', 'group': 'cron', 'enabled': True, 'image': '192.168.2.59:443/openstack.kolla/cron:zed-ubuntu-jammy-aarch64', 'environment': {'KOLLA_LOGROTATE_SCHEDULE': 'daily'}, 'volumes': ['/etc/kolla/cron/:/var/lib/kolla/config_files/:ro', '/etc/localtime:/etc/localtime:ro', '/etc/timezone:/etc/timezone:ro', 'kolla_logs:/var/log/kolla/'], 'dimensions': {}}}) TASK [common : Copy rabbitmq-env.conf to kolla toolbox] ********************************************************************************************************************** changed: [ost61] => (item=/home/ubuntu/labs/ostack/kolla-ansible-zed/share/kolla-ansible/ansible/roles/common/templates/rabbitmq-env.conf.j2) changed: [ost64] => (item=/home/ubuntu/labs/ostack/kolla-ansible-zed/share/kolla-ansible/ansible/roles/common/templates/rabbitmq-env.conf.j2) changed: [ost62] => (item=/home/ubuntu/labs/ostack/kolla-ansible-zed/share/kolla-ansible/ansible/roles/common/templates/rabbitmq-env.conf.j2) changed: [ost63] => (item=/home/ubuntu/labs/ostack/kolla-ansible-zed/share/kolla-ansible/ansible/roles/common/templates/rabbitmq-env.conf.j2) TASK [common : Copy rabbitmq erl_inetrc to kolla toolbox] ******************************************************************************************************************** changed: [ost64] => (item=/home/ubuntu/labs/ostack/kolla-ansible-zed/share/kolla-ansible/ansible/roles/common/templates/erl_inetrc.j2) changed: [ost61] => (item=/home/ubuntu/labs/ostack/kolla-ansible-zed/share/kolla-ansible/ansible/roles/common/templates/erl_inetrc.j2) changed: [ost62] => (item=/home/ubuntu/labs/ostack/kolla-ansible-zed/share/kolla-ansible/ansible/roles/common/templates/erl_inetrc.j2) changed: [ost63] => (item=/home/ubuntu/labs/ostack/kolla-ansible-zed/share/kolla-ansible/ansible/roles/common/templates/erl_inetrc.j2) TASK [common : Check common containers] ************************************************************************************************************************************** changed: [ost62] => (item={'key': 'fluentd', 'value': {'container_name': 'fluentd', 'group': 'fluentd', 'enabled': True, 'image': '192.168.2.59:443/openstack.kolla/fluentd:zed-ubuntu-jammy-aarch64', 'environment': {'KOLLA_CONFIG_STRATEGY': 'COPY_ALWAYS'}, 'volumes': ['/etc/kolla/fluentd/:/var/lib/kolla/config_files/:ro', '/etc/localtime:/etc/localtime:ro', '/etc/timezone:/etc/timezone:ro', 'kolla_logs:/var/log/kolla/', 'fluentd_data:/var/lib/fluentd/data/'], 'dimensions': {}}}) changed: [ost61] => (item={'key': 'fluentd', 'value': {'container_name': 'fluentd', 'group': 'fluentd', 'enabled': True, 'image': '192.168.2.59:443/openstack.kolla/fluentd:zed-ubuntu-jammy-aarch64', 'environment': {'KOLLA_CONFIG_STRATEGY': 'COPY_ALWAYS'}, 'volumes': ['/etc/kolla/fluentd/:/var/lib/kolla/config_files/:ro', '/etc/localtime:/etc/localtime:ro', '/etc/timezone:/etc/timezone:ro', 'kolla_logs:/var/log/kolla/', 'fluentd_data:/var/lib/fluentd/data/'], 'dimensions': {}}}) changed: [ost64] => (item={'key': 'fluentd', 'value': {'container_name': 'fluentd', 'group': 'fluentd', 'enabled': True, 'image': '192.168.2.59:443/openstack.kolla/fluentd:zed-ubuntu-jammy-aarch64', 'environment': {'KOLLA_CONFIG_STRATEGY': 'COPY_ALWAYS'}, 'volumes': ['/etc/kolla/fluentd/:/var/lib/kolla/config_files/:ro', '/etc/localtime:/etc/localtime:ro', '/etc/timezone:/etc/timezone:ro', 'kolla_logs:/var/log/kolla/', 'fluentd_data:/var/lib/fluentd/data/'], 'dimensions': {}}}) changed: [ost63] => (item={'key': 'fluentd', 'value': {'container_name': 'fluentd', 'group': 'fluentd', 'enabled': True, 'image': '192.168.2.59:443/openstack.kolla/fluentd:zed-ubuntu-jammy-aarch64', 'environment': {'KOLLA_CONFIG_STRATEGY': 'COPY_ALWAYS'}, 'volumes': ['/etc/kolla/fluentd/:/var/lib/kolla/config_files/:ro', '/etc/localtime:/etc/localtime:ro', '/etc/timezone:/etc/timezone:ro', 'kolla_logs:/var/log/kolla/', 'fluentd_data:/var/lib/fluentd/data/'], 'dimensions': {}}}) changed: [ost64] => (item={'key': 'kolla-toolbox', 'value': {'container_name': 'kolla_toolbox', 'group': 'kolla-toolbox', 'enabled': True, 'image': '192.168.2.59:443/openstack.kolla/kolla-toolbox:zed-ubuntu-jammy-aarch64', 'environment': {'ANSIBLE_NOCOLOR': '1', 'ANSIBLE_LIBRARY': '/usr/share/ansible'}, 'privileged': True, 'volumes': ['/etc/kolla/kolla-toolbox/:/var/lib/kolla/config_files/:ro', '/etc/localtime:/etc/localtime:ro', '/etc/timezone:/etc/timezone:ro', '/dev/:/dev/', '/run/:/run/:shared', 'kolla_logs:/var/log/kolla/'], 'dimensions': {}}}) changed: [ost61] => (item={'key': 'kolla-toolbox', 'value': {'container_name': 'kolla_toolbox', 'group': 'kolla-toolbox', 'enabled': True, 'image': '192.168.2.59:443/openstack.kolla/kolla-toolbox:zed-ubuntu-jammy-aarch64', 'environment': {'ANSIBLE_NOCOLOR': '1', 'ANSIBLE_LIBRARY': '/usr/share/ansible'}, 'privileged': True, 'volumes': ['/etc/kolla/kolla-toolbox/:/var/lib/kolla/config_files/:ro', '/etc/localtime:/etc/localtime:ro', '/etc/timezone:/etc/timezone:ro', '/dev/:/dev/', '/run/:/run/:shared', 'kolla_logs:/var/log/kolla/'], 'dimensions': {}}}) changed: [ost62] => (item={'key': 'kolla-toolbox', 'value': {'container_name': 'kolla_toolbox', 'group': 'kolla-toolbox', 'enabled': True, 'image': '192.168.2.59:443/openstack.kolla/kolla-toolbox:zed-ubuntu-jammy-aarch64', 'environment': {'ANSIBLE_NOCOLOR': '1', 'ANSIBLE_LIBRARY': '/usr/share/ansible'}, 'privileged': True, 'volumes': ['/etc/kolla/kolla-toolbox/:/var/lib/kolla/config_files/:ro', '/etc/localtime:/etc/localtime:ro', '/etc/timezone:/etc/timezone:ro', '/dev/:/dev/', '/run/:/run/:shared', 'kolla_logs:/var/log/kolla/'], 'dimensions': {}}}) changed: [ost63] => (item={'key': 'kolla-toolbox', 'value': {'container_name': 'kolla_toolbox', 'group': 'kolla-toolbox', 'enabled': True, 'image': '192.168.2.59:443/openstack.kolla/kolla-toolbox:zed-ubuntu-jammy-aarch64', 'environment': {'ANSIBLE_NOCOLOR': '1', 'ANSIBLE_LIBRARY': '/usr/share/ansible'}, 'privileged': True, 'volumes': ['/etc/kolla/kolla-toolbox/:/var/lib/kolla/config_files/:ro', '/etc/localtime:/etc/localtime:ro', '/etc/timezone:/etc/timezone:ro', '/dev/:/dev/', '/run/:/run/:shared', 'kolla_logs:/var/log/kolla/'], 'dimensions': {}}}) changed: [ost61] => (item={'key': 'cron', 'value': {'container_name': 'cron', 'group': 'cron', 'enabled': True, 'image': '192.168.2.59:443/openstack.kolla/cron:zed-ubuntu-jammy-aarch64', 'environment': {'KOLLA_LOGROTATE_SCHEDULE': 'daily'}, 'volumes': ['/etc/kolla/cron/:/var/lib/kolla/config_files/:ro', '/etc/localtime:/etc/localtime:ro', '/etc/timezone:/etc/timezone:ro', 'kolla_logs:/var/log/kolla/'], 'dimensions': {}}}) changed: [ost62] => (item={'key': 'cron', 'value': {'container_name': 'cron', 'group': 'cron', 'enabled': True, 'image': '192.168.2.59:443/openstack.kolla/cron:zed-ubuntu-jammy-aarch64', 'environment': {'KOLLA_LOGROTATE_SCHEDULE': 'daily'}, 'volumes': ['/etc/kolla/cron/:/var/lib/kolla/config_files/:ro', '/etc/localtime:/etc/localtime:ro', '/etc/timezone:/etc/timezone:ro', 'kolla_logs:/var/log/kolla/'], 'dimensions': {}}}) changed: [ost64] => (item={'key': 'cron', 'value': {'container_name': 'cron', 'group': 'cron', 'enabled': True, 'image': '192.168.2.59:443/openstack.kolla/cron:zed-ubuntu-jammy-aarch64', 'environment': {'KOLLA_LOGROTATE_SCHEDULE': 'daily'}, 'volumes': ['/etc/kolla/cron/:/var/lib/kolla/config_files/:ro', '/etc/localtime:/etc/localtime:ro', '/etc/timezone:/etc/timezone:ro', 'kolla_logs:/var/log/kolla/'], 'dimensions': {}}}) changed: [ost63] => (item={'key': 'cron', 'value': {'container_name': 'cron', 'group': 'cron', 'enabled': True, 'image': '192.168.2.59:443/openstack.kolla/cron:zed-ubuntu-jammy-aarch64', 'environment': {'KOLLA_LOGROTATE_SCHEDULE': 'daily'}, 'volumes': ['/etc/kolla/cron/:/var/lib/kolla/config_files/:ro', '/etc/localtime:/etc/localtime:ro', '/etc/timezone:/etc/timezone:ro', 'kolla_logs:/var/log/kolla/'], 'dimensions': {}}}) TASK [common : Creating log volume] ****************************************************************************************************************************************** ok: [ost64] ok: [ost61] ok: [ost62] ok: [ost63] TASK [common : Link kolla_logs volume to /var/log/kolla] ********************************************************************************************************************* changed: [ost64] changed: [ost61] changed: [ost62] changed: [ost63] TASK [common : Flush handlers] *********************************************************************************************************************************************** RUNNING HANDLER [common : Restart fluentd container] ************************************************************************************************************************* changed: [ost63] changed: [ost64] changed: [ost62] changed: [ost61] RUNNING HANDLER [common : Restart kolla-toolbox container] ******************************************************************************************************************* changed: [ost62] changed: [ost61] changed: [ost63] changed: [ost64] RUNNING HANDLER [common : Initializing toolbox container using normal user] ************************************************************************************************** fatal: [ost62]: FAILED! => {"changed": false, "cmd": ["docker", "exec", "-t", "kolla_toolbox", "ansible", "--version"], "delta": "0:00:00.064846", "end": "2023-11-19 21:46:19.187519", "msg": "non-zero return code", "rc": 1, "start": "2023-11-19 21:46:19.122673", "stderr": "Error response from daemon: Container 34d4dcc029d807cb70061bcab3f21e1980ec2541b6e116764dab22917d1ba602 is restarting, wait until the container is running", "stderr_lines": ["Error response from daemon: Container 34d4dcc029d807cb70061bcab3f21e1980ec2541b6e116764dab22917d1ba602 is restarting, wait until the container is running"], "stdout": "", "stdout_lines": []} fatal: [ost61]: FAILED! => {"changed": false, "cmd": ["docker", "exec", "-t", "kolla_toolbox", "ansible", "--version"], "delta": "0:00:00.170840", "end": "2023-11-19 21:46:19.383867", "msg": "non-zero return code", "rc": 1, "start": "2023-11-19 21:46:19.213027", "stderr": "Error response from daemon: Container fbe3a84809b7b5e23dbc84e896735bcc8c72fbd300b53637061bd305a6bd45eb is restarting, wait until the container is running", "stderr_lines": ["Error response from daemon: Container fbe3a84809b7b5e23dbc84e896735bcc8c72fbd300b53637061bd305a6bd45eb is restarting, wait until the container is running"], "stdout": "", "stdout_lines": []} fatal: [ost63]: FAILED! => {"changed": false, "cmd": ["docker", "exec", "-t", "kolla_toolbox", "ansible", "--version"], "delta": "0:00:00.422087", "end": "2023-11-19 21:46:19.579842", "msg": "non-zero return code", "rc": 1, "start": "2023-11-19 21:46:19.157755", "stderr": "Error response from daemon: Container 2b8c1c77cd75d8780ad358407178d0e8d431e22885fc0f58ae1e08e746928d9d is restarting, wait until the container is running", "stderr_lines": ["Error response from daemon: Container 2b8c1c77cd75d8780ad358407178d0e8d431e22885fc0f58ae1e08e746928d9d is restarting, wait until the container is running"], "stdout": "", "stdout_lines": []} fatal: [ost64]: FAILED! => {"changed": false, "cmd": ["docker", "exec", "-t", "kolla_toolbox", "ansible", "--version"], "delta": "0:00:00.693630", "end": "2023-11-19 21:46:19.859925", "msg": "non-zero return code", "rc": 1, "start": "2023-11-19 21:46:19.166295", "stderr": "unable to upgrade to tcp, received 409", "stderr_lines": ["unable to upgrade to tcp, received 409"], "stdout": "", "stdout_lines": []}

1 1

[release] Release countdown for week R-19, Nov 20-24
by Herve Beraud 17 Nov '23

17 Nov '23

Development Focus ----------------- We are now past the Caracal-1 milestone. Teams should now be focused on feature development. General Information ------------------- Our next milestone in this development cycle will be Caracal-2, on January 11, 2024. This milestone is when we freeze the list of deliverables that will be included in the 2024.1 Caracal final release, so if you plan to introduce new deliverables in this release, please propose a change to add an empty deliverable file in the deliverables/caracal directory of the openstack/releases repository. Now is also generally a good time to look at bugfixes that were introduced in the master branch that might make sense to be backported and released in a stable release. If you have any question around the OpenStack release process, feel free to ask on this mailing-list or on the #openstack-release channel on IRC. Upcoming Deadlines & Dates -------------------------- Caracal-2 Milestone: January 11, 2024 -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/

1 0

[kolla-ansible][cinder]Backend problems
by Franck VEDEL 17 Nov '23

17 Nov '23

Hi, I come back to a point that caused me a big problem, because I would like to understand something that I did not do correctly. I have a lab with an Openstack (2023.1, kolla-ansible, ubuntu 22-04, 2 nodes) and to show that it works, I start with an LVM backend. Everything is working. I'm adding a Ceph (Pacific) cluster and want to switch from LVM backend to CEPH. If we edit the globals.yml file and make the changes, then the "deploy" command, the LVM backend is still in the list of backends. If we restart the cinder containers, same thing. If we do "stop" then "deploy": same thing. I had to do "stop", "destroy", "deploy" to no longer have the LVM backend but only the CEPH backend. My question is therefore the following: what should we do in this situation? Is there a solution to easily no longer take into account a backend? Thansk for your help Franck

2 2

[kolla][masakari] Deprecating Masakari Support in Kolla-Ansible
by Michał Nasiadka 17 Nov '23

17 Nov '23

Hello, Since the Kolla-Ansible CI jobs for Masakari using 2023.2 have been failing [1] due to some time - we are deprecating Masakari support in Kolla/Kolla-Ansible. As I can see - even the OpenStack bot patches have not been merged in 2023.2 branch [2]. If there is anybody willing to go the extra mile of reviving Masakari stable/2023.2 branch - please reach out - ideally by commenting on the deprecation patch in Gerrit [3]. [1]: https://zuul.opendev.org/t/openstack/builds?job_name=kolla-ansible-ubuntu-m… [2]: https://review.opendev.org/q/project:openstack/masakari+is:open+branch:stab… [3]: https://review.opendev.org/c/openstack/kolla-ansible/+/901193

1 0

[neutron] Drivers meeting cancelled next two weeks
by Brian Haley 16 Nov '23

16 Nov '23

Hello Neutrinos: Due to the lack of agenda, Friday's drivers meeting is cancelled. Have a nice weekend! Also canceling next week's since with the US Thanksgiving holiday we will likely not have quorum. -Brian

1 0

openstack: how can you remove volumen from postgres database on openstack controller?
by kjme001＠gmail.com 16 Nov '23

16 Nov '23

Hello, I have a problem with the volume, I can't delete it, it's still in the deleting state how can you remove volumen from postgres database on openstack controller? Thank you for your help

1 0

[neutron] How to divide a provider network into address segments for different projects to use
by 韩光宇 16 Nov '23

16 Nov '23

Hi, I have a victoria cluster. And all node have two nic. Same to neutron doc[1], one nic is management interface, other is provider interface. Assuming provider network is same as the example in the Neutron deployment doc, which is the 203.0.113.0/24 with gateway 203.0.113.1. I hope use different address segment for different project. For example: project_A: 203.0.113.11,203.0.113.30 project_B: 203.0.113.31,203.0.113.60 project_C: 203.0.113.61,203.0.113.90 I am currently considering one method: Creating different VLAN networks for different projects, but it requires configuring the physical switch ports as trunks. Is there any other way to achieve this? I would prefer to use a flat network. I will appreciate for any help. Best regards, Han Guangyu [1]: https://docs.openstack.org/neutron/latest/install/environment-networking-ob…

1 0

[Glance] Cancelling Weekly Meeting - 16th Nov
by Mridula Joshi 16 Nov '23

16 Nov '23

Hello All, Cancelling today's weekly meeting as most of the team members are not around for the meeting. See you next week ! Thanks, Mridula Joshi

1 0

[cinder] Festival of XS reviews (17 Nov)
by Rajat Dhasmana 16 Nov '23

16 Nov '23

Hello Argonauts, We will be having our monthly festival of reviews tomorrow i.e. 17th November (Friday) from 1400-1600 UTC. Following are the details: Date: 17th November, 2023 Time: 1400-1600 UTC Meeting link: https://meet.google.com/jqg-eigw-rku etherpad: https://etherpad.opendev.org/p/cinder-festival-of-reviews Thanks Rajat Dhasmana

1 0

openstack: aborted: Failed to allocate the network(s), not rescheduling.].
by kjme001＠gmail.com 16 Nov '23

16 Nov '23

when creating a new instance I get an error: aborted: Failed to allocate the network(s), not rescheduling.]. Can anyone tell me where to look for the problem? Regards

3 2

RE: openstack-discuss Digest, Vol 61, Issue 61- Nova- Compute Service Issue
by Asma Naz Shariq 16 Nov '23

16 Nov '23

Hello Openstack Team, I have deployed openstack zed release with multinode cluster through kolla-ansible controller with 1x storage, 1x compute node. The nova-compute service failed to register itself on the compute-host as attached; due to which there encounters an error of no availability zone for an instance to launch. Can any one guide me how to resolve this nova service issues. -----Original Message----- From: openstack-discuss-request(a)lists.openstack.org <openstack-discuss-request(a)lists.openstack.org> Sent: Thursday, November 16, 2023 2:15 AM To: openstack-discuss(a)lists.openstack.org Subject: openstack-discuss Digest, Vol 61, Issue 61 Send openstack-discuss mailing list submissions to openstack-discuss(a)lists.openstack.org To subscribe or unsubscribe via email, send a message with subject or body 'help' to openstack-discuss-request(a)lists.openstack.org You can reach the person managing the list at openstack-discuss-owner(a)lists.openstack.org When replying, please edit your Subject line so it is more specific than "Re: Contents of openstack-discuss digest..." Today's Topics: 1. Re: [kolla-ansible][cinder-schedule] Problem between Ceph and cinder-schedule (Franck VEDEL) 2. Re: Re: [keystone] temporary workaround for deprecated _member_ role? (Michael Knox) 3. openstack: aborted: Failed to allocate the network(s), not rescheduling.]. (kjme001(a)gmail.com) ---------------------------------------------------------------------- Message: 1 Date: Wed, 15 Nov 2023 21:33:15 +0100 From: Franck VEDEL <franck.vedel(a)univ-grenoble-alpes.fr> Subject: Re: [kolla-ansible][cinder-schedule] Problem between Ceph and cinder-schedule To: Alan Bishop <abishop(a)redhat.com> Cc: openstack-discuss(a)lists.openstack.org Message-ID: <63B87539-94A5-4CA5-A6CA-9CB67124EAEF(a)univ-grenoble-alpes.fr> Content-Type: multipart/alternative; boundary="Apple-Mail=_880E21AE-FC09-43C2-8C33-2E2670B2D035" Alan ! Thanks a lot…. I had doubts about the last config because I only wanted to use the Ceph cluster, but cinder-scheduler was looking for a backend which it could not find. But the cluster was functional so: kolla-ansible -i multinode stop …..destroy…. deploy…. post-deploy…. init-runonce…. Et VOILA. This reset all configurations, adds and rollbacks to 0. And it works ! Thanks!! Franck > Le 15 nov. 2023 à 15:43, Alan Bishop <abishop(a)redhat.com> a écrit : > > > > On Wed, Nov 15, 2023 at 6:32 AM Franck VEDEL <franck.vedel(a)univ-grenoble-alpes.fr <mailto:franck.vedel@univ-grenoble-alpes.fr>> wrote: >> Hi Pierre >> Maybe I found something: >> In the container « cinder_volume », there is a file >> /etc/ceph/ceph.conf But, no /etc/ceph/ceph.conf file in « cinder_scheduler » container. >> I think it’s my problem !! > > Sorry, but the scheduler does not require access to /etc/ceph files. I > think you need to review the scheduler logs to see why it concluded > there are no available c-vol backends. Since you are able to create a volume, I assume the ceph backend is "up," but it would also be good to verify it remains up, especially during the time when you're trying to create a volume from an image. > > Alan > >> but why ? >> What is the option in globals.yml ? >> Or maybe there is something to add in /etc/kolla/config/cinder/ >> >> I have this (the last try): >> >> >> enable_cinder: "yes" >> enable_cinder_backup: "no" >> enable_cinder_backend_lvm: "no" >> external_ceph_cephx_enabled: "yes" >> >> # Glance >> ceph_glance_keyring: "ceph.client.glance.keyring" >> ceph_glance_user: "glance" >> ceph_glance_pool_name: "images" >> # Cinder >> ceph_cinder_keyring: "ceph.client.cinder.keyring" >> ceph_cinder_user: "cinder" >> ceph_cinder_pool_name: "volumes" >> >> #ceph_cinder_backup_keyring: "ceph.client.cinder-backup.keyring" >> #ceph_cinder_backup_keyring: "ceph.client.cinder.keyring" >> #ceph_cinder_backup_user: "cinder" >> #ceph_cinder_backup_user: "cinder-backup" >> #ceph_cinder_backup_pool_name: "backups" >> # Nova >> #ceph_nova_keyring: "{{ ceph_cinder_keyring }}" >> ceph_nova_keyring: "ceph.client.nova.keyring" >> ceph_nova_user: "nova" >> ceph_nova_pool_name: "vms" >> >> # Configure image backend. >> glance_backend_ceph: "yes" >> >> # Enable / disable Cinder backends >> cinder_backend_ceph: "yes" >> >> # Nova - Compute Options >> ######################## >> nova_backend_ceph: "yes" >> >> >> Thanks a lot >> >> >> >> Franck >> >> >>> Le 15 nov. 2023 à 09:54, Pierre Riteau <pierre(a)stackhpc.com <mailto:pierre@stackhpc.com>> a écrit : >>> >>> Hi Franck, >>> >>> It would help if you could share more details about the error (check both cinder-scheduler and cinder-volume logs). >>> >>> Do you have capabilities to access the images pool (presumably what is used by Glance) on your client.cinder user? >>> >>> Best wishes, >>> Pierre Riteau (priteau) >>> >>> On Wed, 15 Nov 2023 at 09:50, Franck VEDEL <franck.vedel(a)univ-grenoble-alpes.fr <mailto:franck.vedel@univ-grenoble-alpes.fr>> wrote: >>>> Good morning, >>>> I'll come back for help, thanks in advance. >>>> I am testing an Openstack 2023.1/Ceph Pacific environment. >>>> The cluster works, and from Horizon for example, if I create a simple volume, it works normally, it is placed in the "volumes" pool in the Ceph cluster, exactly as expected. >>>> >>>> if I create a volume from an image, ERROR cinder-schedule. >>>> In the cinder_volume and cinder_scheduler containers, same settings in cinder.conf. >>>> >>>> Before continuing to find out what the error could be, could there be a problem between Antelope (2023.1) and Ceph Pacific? (I'm on Ubuntu 22.04). >>>> >>>> Franck >>>> >>>> >>>> >>

1 0

[kolla-ansible][cinder-schedule] Problem between Ceph and cinder-schedule
by Franck VEDEL 15 Nov '23

15 Nov '23

Good morning, I'll come back for help, thanks in advance. I am testing an Openstack 2023.1/Ceph Pacific environment. The cluster works, and from Horizon for example, if I create a simple volume, it works normally, it is placed in the "volumes" pool in the Ceph cluster, exactly as expected. if I create a volume from an image, ERROR cinder-schedule. In the cinder_volume and cinder_scheduler containers, same settings in cinder.conf. Before continuing to find out what the error could be, could there be a problem between Antelope (2023.1) and Ceph Pacific? (I'm on Ubuntu 22.04). Franck

4 8

[cinder][ops] Deprecating vmdk volume driver
by Takashi Kajinami 15 Nov '23

15 Nov '23

Hello, # This is similar to my previous email[1] but this one is for a different component. Cinder has provided the following two volume drivers to manage volumes in VMWare product. VMwareVcVmdkDriver for VMWare vCenter server VMwareVStorageObjectDriver for VMWare VStorageObject These drivers may be useful for OpenStack deployments using VMWare hypervisors. However the vmware api virt driver is being removed during this cycle, and this use case is no longer supported. I'm sending this email to hear if anyone is using these volume drivers WITHOUT the vmware api virt driver. If no, then I'll propose deprecating these drivers in 2024.1 release, and remove these in 2024.2. In case you have any concerns (or agree with the plan) then please reply to this thread (or to me directly if needed). Thank you, Takashi [1] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack…

1 0

Cinder Bug Report 2023-11-15
by Jon Bernard 15 Nov '23

15 Nov '23

Hello Argonauts, Cinder Bug Meeting Etherpad <https://etherpad.opendev.org/p/cinder-bug-squad-meeting> There are exactly 0 new bugs this week, so no bug meeting today! Thanks, -- Jon

1 0

[largescale-sig] Next meeting: Nov 15, 8utc
by Thierry Carrez 15 Nov '23

15 Nov '23

Hi everyone, The Large Scale SIG will be meeting this Wednesday in #openstack-operators on OFTC IRC, at 8UTC, our EU+APAC-friendly time. Note that in some parts of the world, DST means that that meeting will be one hour earlier than usual. You can doublecheck how that UTC time translates locally at: https://www.timeanddate.com/worldclock/fixedtime.html?iso=20231115T08 Feel free to add topics to the agenda: https://etherpad.opendev.org/p/large-scale-sig-meeting Thanks, -- Thierry Carrez (ttx)

1 1

[tc][releases][ptl][all] 'Unmaintained' branch update
by Brian Rosmaita 15 Nov '23

15 Nov '23

Hello Stackers, I'm sure you saw Elod's email eariler this week [0] calling for final releases from stable/yoga in preparation for its transition to 'Unmaintained' status. In addition to stable/yoga, the TC resolution [1] specified that the three most recent Extended Maintenance branches would also transition to 'Unmaintained', and branches older than that can be deleted. The TC and the release team have had some discussions over the past two weeks [2,3,4,5], and we are closing in on a consensus on how the proposal described in the 2023-07024 resolution should work. This is how it's looking: (1) branches ineligible for Unmaintained status These are stable/ussuri and stable/train. The release team will use the current tooling to generate the patches proposing the ussuri-eol and train-eol tags; teams will have a grace period to get any final changes merged; after the -eol tags are made, the branches will be deleted. (2) transitioning a branch out of Maintained status This is stable/yoga. Elod has already sent the reminder email [0] for project teams to propose a release patch if there's a reason to do a final release. The release team is working on tooling that will mark the HEAD of stable/yoga with yoga-eom (for "end of maintenance"), delete the stable/yoga branch, and then cut the unmaintained/yoga branch based on the yoga-eom tag. There will be a grace period for the community to decide whether the branch should be kept in Unmaintained status or deleted, as described in [1]. Eventually (either when the grace period expires, or later, when there is no longer interest in keeping unmaintained/yoga around), the HEAD will be tagged yoga-eol and the branch deleted. (3) transitioning current eligible Extended Maintenance branches to Unmaintained status These are stable/xena, stable/wallaby, and stable/victoria. Ericsson has expressed interest in keeping victoria in Unmaintained status. Subject to the guidelines in [1], wallaby, xena, and yoga will therefore be kept in Unmaintained status as well. Eventually, consistent with [1], these branches will be tagged $series-eol and deleted. For more details about (1)-(3) above, see the discussion in [6]. The current patch to the Project Team Guide [7] will be updated shortly to reflect the discussion in [6]. (4) amendment to the Unmaintained branch resolution The original plan in [1] was for there to be a $project-unmaintained-core Gerrit group for each OpenStack project that would have +2/+W powers on the unmaintained/$series branches. This would entail changes in the Gerrit ACL config files for all projects plus populating all the Gerrit groups. I've put up an amendment [8] to the resolution that would instead establish a single openstack-unmaintained-core Gerrit group that would have +2/+W powers over all the unmaintained/* branches in the openstack namespace. Projects that would prefer to stick with the original proposal and have tighter control over their unmaintained/* branches can still do so; they'll have to patch the appropriate Gerrit ACL config file(s) for their project and populate the $project-unmaintained-core group. Note that except for the Amendment mentioned in (4), this email only provides details about how [1] is to be implemented; it does not change the nature of the Unmaintained branch proposal that was adopted by the TC back in July 2023. If you have questions or comments, please reply to the openstack-discuss mailing list, or leave comments on [7] and [8]. [0] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [1] https://governance.openstack.org/tc/resolutions/20230724-unmaintained-branc… [2] https://meetings.opendev.org/meetings/tc/2023/tc.2023-10-31-18.00.log.html#… [3] https://meetings.opendev.org/meetings/tc/2023/tc.2023-11-07-18.00.log.html#… video meeting recording: https://www.youtube.com/watch?v=qv6nSRhfV_M [4] https://meetings.opendev.org/meetings/releaseteam/2023/releaseteam.2023-11-… [5] https://meetings.opendev.org/meetings/tc/2023/tc.2023-11-14-18.00.log.html#… [6] https://etherpad.opendev.org/p/early-caracal-unmaintained-transition [7] https://review.opendev.org/c/openstack/project-team-guide/+/897505 [8] https://review.opendev.org/c/openstack/governance/+/900940

1 0

Discuss i18n IRC Meeting date
by Seongsoo Cho 15 Nov '23

15 Nov '23

Hi i18n Team. It's time to update our regular meeting date. This is the current meeting date in the doc, but actually, there are no meeting. [1] Every odd week: 16:00 UTC, Thursday Every even week: 07:00 UTC, Thursday So I want to set a new meeting date. For now, I'd like to have a monthly IRC meeting. Please let me know what days and times work best for you. Survey form: https://forms.gle/SZUFmwJS6QTs4akNA Best Regards Seongsoo [1] https://docs.openstack.org/i18n/latest/i18n_team_meeting.html -- Seongsoo Cho OpenStack Korea User Group / Community Leader IRC #seongsoocho

1 0

[ptls][PTG] OpenInfra Live: PTG Recap
by Kristin Barrientos 15 Nov '23

15 Nov '23

Hi Everyone, I wanted to reach out to see if any PTLs were interested in providing their highlights from last month’s PTG in an OpenInfra Live[1] episode on Thursday, December 7 at 1500 UTC. Ideally, we would like to have 4-6 projects represented (OpenStack, Kata, and/or StarlingX). A previous example of a PTG recap episode can be found here[2]. Please let me know if you’re interested and I can provide the next steps. If you would like to provide an update but that time doesn’t work for you, please share a recording with me and I can get it added. Thanks, Kristin Barrientos Marketing Coordinator OpenInfra Foundation [1] openinfra.dev/live [2] https://www.youtube.com/watch?v=yIt4dJvTQVg&t=231s

7 11

[swift]: Swiftstack Zone/DC decommission
by Muralikrishna Gutha 15 Nov '23

15 Nov '23

Hello Team, We currently utilize SwiftStack Object Storage to meet our object storage requirements, and our cluster has a Primary Zone/Secondary Zone topology. Recently, we attempted to remove/disable the Secondary Zone. However, upon doing so, we observed a concerning trend: the storage utilization in the Primary Zone rapidly increased, reaching approximately 90%. This surge was particularly notable due to the use of the 'tworeplicaperregion' policy. In an attempt to address this, we brought the Secondary Zone nodes back online, expecting Swift to redistribute the data back to these nodes and alleviate the storage strain on the Primary Zone. Despite reintegrating the Secondary Zone nodes into the cluster, we continue to witness a persistent rise in storage utilization within the Primary Zone. Interestingly, two out of the three nodes in the Primary Zone seem to bear the brunt of the data and tokens, while the third node is not receiving a proportional distribution of data. At this juncture, we are seeking guidance on potential configurations or adjustments, such as specific knobs, that could expedite the rebalance process. Our objective is to efficiently distribute the data across nodes, particularly in the Primary Zone, and optimize the overall storage utilization in our SwiftStack Object Storage cluster. Any insights or recommendations on tuning parameters for a swifter rebalance would be highly appreciated. Thanks, Murali -- This message may contain confidential and/or privileged information. If you are not the addressee or authorized to receive this on behalf of the addressee you must not use, copy, disclose or take action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply email and delete this message. Thank you.

3 7

[tc] Next TC Meeting will be Nov 28 (2023-11-21 meeting cancelled)
by Jay Faulkner 14 Nov '23

14 Nov '23

Hi all, The Technical Committee will not be meeting next week. The meeting was cancelled due to American Thanksgiving. Our next meeting will be November 28, 2023. Thanks, Jay Faulkner OpenStack TC Chair

1 0

Implementation review day will be Nov 15
by Sylvain Bauza 14 Nov '23

14 Nov '23

Hey folks, We just agreed during today's meeting on having our first implementation review day this week. In order to make this day efficient, I proposed an etherpad for tracking our efforts https://etherpad.opendev.org/p/nova-caracal-status Shall you want the nova community to look at your changes, you should please add them into the etherpad in the very last section ('proposed bugfixes') and then me and a few other cores will curate that list. Tomorrow, we'll scrub the list of approved blueprints's patches and the bugfixes' ones that got approved as a review priority and we'll iterate over them. Please try to look at your Gerrit emails, you could get comments ;-) Thanks, -Sylvain

1 1

the volume cannot be deleted, it is in deleting state
by kjme001＠gmail.com 14 Nov '23

14 Nov '23

hello, I'm having trouble deleting volumes in OpenStackl from a machine that no longer exists. How to execute cinder command reset-state --attach-status detached volume_name then I execute the command: # cinder delete volume_name Delete for volume cifar-10-copy failed: Invalid volume: Volume status must be available or error or error_restoring or error_extending or error_managing and must not be migrating, attached, belong to a group, have snapshots or be disassociated from snapshots after volume transfer. (HTTP 400) (Request-ID: req-b2d3f2e4-7855-4831-a68d-f6710cc97e94) ERROR: Unable to delete any of the specified volumes. then executes the command: cinder reset-state --state available volume_name and cinder delete volume_name Request to delete volume volume_name has been accepted. when I check: openstack volume show volume_name is: | status | deleting The volume is not being deleted, it is in the deleting state. This is a volume from a non-existent node, but the volume is still visible in OpenStack and cannot be deleted because it is in the deleting state. Maybe someone can tell me how to remove such a volume? Regards

1 0

cinder-manage service remove cinder-volume xxxx@lvm-ssd - error
by kjme001＠gmail.com 14 Nov '23

14 Nov '23

cinder-manage service remove cinder-volume xxxx@lvm-ssd /usr/lib/python3/dist-packages/cinder/db/sqlalchemy/models.py:152: SAWarning: implicitly coercing SELECT object to scalar subquery; please use the .scalar_subquery() method to produce a scalar subquery. last_heartbeat = column_property( /usr/lib/python3/dist-packages/cinder/db/sqlalchemy/models.py:160: SAWarning: implicitly coercing SELECT object to scalar subquery; please use the .scalar_subquery() method to produce a scalar subquery. num_hosts = column_property( /usr/lib/python3/dist-packages/cinder/db/sqlalchemy/models.py:169: SAWarning: implicitly coercing SELECT object to scalar subquery; please use the .scalar_subquery() method to produce a scalar subquery. num_down_hosts = column_property( 2023-11-11 14:34:00.928 898920 CRITICAL cinder [req-525973bb-e026-4999-95af-d852aa137066 - - - - -] Unhandled error: oslo_db.exception.DBNonExistentDatabase: (sqlite3.OperationalError) unable to open database file (Background on this error at: https://sqlalche.me/e/14/e3q8) 2023-11-11 14:34:00.928 898920 ERROR cinder Traceback (most recent call last): 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 3250, in _wrap_pool_connect 2023-11-11 14:34:00.928 898920 ERROR cinder return fn() 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 310, in connect 2023-11-11 14:34:00.928 898920 ERROR cinder return _ConnectionFairy._checkout(self) 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 868, in _checkout 2023-11-11 14:34:00.928 898920 ERROR cinder fairy = _ConnectionRecord.checkout(pool) 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 476, in checkout 2023-11-11 14:34:00.928 898920 ERROR cinder rec = pool._do_get() 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/sqlalchemy/pool/impl.py", line 256, in _do_get 2023-11-11 14:34:00.928 898920 ERROR cinder return self._create_connection() 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 256, in _create_connection 2023-11-11 14:34:00.928 898920 ERROR cinder return _ConnectionRecord(self) 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 371, in __init__ 2023-11-11 14:34:00.928 898920 ERROR cinder self.__connect() 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 665, in __connect 2023-11-11 14:34:00.928 898920 ERROR cinder with util.safe_reraise(): 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ 2023-11-11 14:34:00.928 898920 ERROR cinder compat.raise_( 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ 2023-11-11 14:34:00.928 898920 ERROR cinder raise exception 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 661, in __connect 2023-11-11 14:34:00.928 898920 ERROR cinder self.dbapi_connection = connection = pool._invoke_creator(self) 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/sqlalchemy/engine/create.py", line 590, in connect 2023-11-11 14:34:00.928 898920 ERROR cinder return dialect.connect(*cargs, **cparams) 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/sqlalchemy/engine/default.py", line 597, in connect 2023-11-11 14:34:00.928 898920 ERROR cinder return self.dbapi.connect(*cargs, **cparams) 2023-11-11 14:34:00.928 898920 ERROR cinder sqlite3.OperationalError: unable to open database file 2023-11-11 14:34:00.928 898920 ERROR cinder 2023-11-11 14:34:00.928 898920 ERROR cinder The above exception was the direct cause of the following exception: 2023-11-11 14:34:00.928 898920 ERROR cinder 2023-11-11 14:34:00.928 898920 ERROR cinder Traceback (most recent call last): 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/bin/cinder-manage", line 10, in <module> 2023-11-11 14:34:00.928 898920 ERROR cinder sys.exit(main()) 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/cinder/cmd/manage.py", line 1069, in main 2023-11-11 14:34:00.928 898920 ERROR cinder fn(**fn_kwargs) 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/cinder/cmd/manage.py", line 711, in remove 2023-11-11 14:34:00.928 898920 ERROR cinder svc = objects.Service.get_by_args(ctxt, host_name, binary) 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/cinder/objects/service.py", line 133, in get_by_args 2023-11-11 14:34:00.928 898920 ERROR cinder db_service = db.service_get(context, host=host, binary=binary_key) 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/cinder/db/api.py", line 270, in service_get 2023-11-11 14:34:00.928 898920 ERROR cinder return IMPL.service_get(context, service_id, backend_match_level, 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/cinder/db/sqlalchemy/api.py", line 168, in wrapper 2023-11-11 14:34:00.928 898920 ERROR cinder return f(*args, **kwargs) 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/cinder/db/sqlalchemy/api.py", line 745, in service_get 2023-11-11 14:34:00.928 898920 ERROR cinder query = _service_query(context, backend_match_level=backend_match_level, 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/cinder/db/sqlalchemy/api.py", line 686, in _service_query 2023-11-11 14:34:00.928 898920 ERROR cinder query = model_query(context, models.Service, session=session, 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/cinder/db/sqlalchemy/api.py", line 259, in model_query 2023-11-11 14:34:00.928 898920 ERROR cinder session = kwargs.get('session') or get_session() 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/cinder/db/sqlalchemy/api.py", line 95, in get_session 2023-11-11 14:34:00.928 898920 ERROR cinder return main_context_manager._factory.get_legacy_facade().get_session( 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 357, in get_legacy_facade 2023-11-11 14:34:00.928 898920 ERROR cinder self._start() 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 504, in _start 2023-11-11 14:34:00.928 898920 ERROR cinder self._setup_for_connection( 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 529, in _setup_for_connection 2023-11-11 14:34:00.928 898920 ERROR cinder engine = engines.create_engine( 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/debtcollector/renames.py", line 43, in decorator 2023-11-11 14:34:00.928 898920 ERROR cinder return wrapped(*args, **kwargs) 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/oslo_db/sqlalchemy/engines.py", line 211, in create_engine 2023-11-11 14:34:00.928 898920 ERROR cinder test_conn = _test_connection(engine, max_retries, retry_interval) 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/oslo_db/sqlalchemy/engines.py", line 386, in _test_connection 2023-11-11 14:34:00.928 898920 ERROR cinder return engine.connect() 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 3204, in connect 2023-11-11 14:34:00.928 898920 ERROR cinder return self._connection_cls(self, close_with_result=close_with_result) 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 96, in __init__ 2023-11-11 14:34:00.928 898920 ERROR cinder else engine.raw_connection() 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 3283, in raw_connection 2023-11-11 14:34:00.928 898920 ERROR cinder return self._wrap_pool_connect(self.pool.connect, _connection) 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 3253, in _wrap_pool_connect 2023-11-11 14:34:00.928 898920 ERROR cinder Connection._handle_dbapi_exception_noconnection( 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 2098, in _handle_dbapi_exception_noconnection 2023-11-11 14:34:00.928 898920 ERROR cinder util.raise_(newraise, with_traceback=exc_info[2], from_=e) 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ 2023-11-11 14:34:00.928 898920 ERROR cinder raise exception 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 3250, in _wrap_pool_connect 2023-11-11 14:34:00.928 898920 ERROR cinder return fn() 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 310, in connect 2023-11-11 14:34:00.928 898920 ERROR cinder return _ConnectionFairy._checkout(self) 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 868, in _checkout 2023-11-11 14:34:00.928 898920 ERROR cinder fairy = _ConnectionRecord.checkout(pool) 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 476, in checkout 2023-11-11 14:34:00.928 898920 ERROR cinder rec = pool._do_get() 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/sqlalchemy/pool/impl.py", line 256, in _do_get 2023-11-11 14:34:00.928 898920 ERROR cinder return self._create_connection() 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 256, in _create_connection 2023-11-11 14:34:00.928 898920 ERROR cinder return _ConnectionRecord(self) 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 371, in __init__ 2023-11-11 14:34:00.928 898920 ERROR cinder self.__connect() 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 665, in __connect 2023-11-11 14:34:00.928 898920 ERROR cinder with util.safe_reraise(): 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ 2023-11-11 14:34:00.928 898920 ERROR cinder compat.raise_( 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ 2023-11-11 14:34:00.928 898920 ERROR cinder raise exception 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 661, in __connect 2023-11-11 14:34:00.928 898920 ERROR cinder self.dbapi_connection = connection = pool._invoke_creator(self) 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/sqlalchemy/engine/create.py", line 590, in connect 2023-11-11 14:34:00.928 898920 ERROR cinder return dialect.connect(*cargs, **cparams) 2023-11-11 14:34:00.928 898920 ERROR cinder File "/usr/lib/python3/dist-packages/sqlalchemy/engine/default.py", line 597, in connect 2023-11-11 14:34:00.928 898920 ERROR cinder return self.dbapi.connect(*cargs, **cparams) 2023-11-11 14:34:00.928 898920 ERROR cinder oslo_db.exception.DBNonExistentDatabase: (sqlite3.OperationalError) unable to open database file 2023-11-11 14:34:00.928 898920 ERROR cinder (Background on this error at: https://sqlalche.me/e/14/e3q8) 2023-11-11 14:34:00.928 898920 ERROR cinder

3 4

(no subject)
by mgutha＠liveperson.com 14 Nov '23

14 Nov '23

I have observed an imbalance in the cluster based on the object.builder output. Is this possibly due to an ongoing rebalance process? Additionally, I'd like to confirm my understanding of 'min_part_hours.' Is it accurate to say that this parameter represents the number of hours an object needs to reside on a disk before it's considered for movement to another device during a rebalance? According to the Swift documentation, the default value is 24 hours. I'm curious if this value is applied to data written specifically by the rebalance process or if it also includes data written by other Swift services. object.builder, build version 775, id 61b2166cb8684b2b80d6fc370cef37a6 65536 partitions, 2.000000 replicas, 2 regions, 2 zones, 95 devices, 38.89 balance, 0.00 dispersion The minimum number of hours before a partition can be reassigned is 0 (0:00:00 remaining) The overload factor is 10.00% (0.100000) Ring file object.ring.gz not found, probably it hasn't been written yet Devices: id region zone ip address:port replication ip:port name weight partitions balance flags meta 129 1 1 182.16.128.36:6000 182.16.128.36:6003 d129 1.00 2 38.89 130 1 1 182.16.128.36:6006 182.16.128.36:6003 d130 1.00 2 38.89 131 1 1 182.16.128.36:6007 182.16.128.36:6003 d131 1.00 2 38.89 132 1 1 182.16.128.36:6008 182.16.128.36:6003 d132 1.00 2 38.89 133 1 1 182.16.128.36:6009 182.16.128.36:6003 d133 1.00 2 38.89 134 1 1 182.16.128.36:6010 182.16.128.36:6003 d134 1.00 1 -30.56 135 1 1 182.16.128.36:6011 182.16.128.36:6003 d135 1.00 1 -30.56 136 1 1 182.16.128.36:6012 182.16.128.36:6003 d136 1.00 1 -30.56 137 1 1 182.16.128.36:6013 182.16.128.36:6003 d137 1.00 1 -30.56 138 1 1 182.16.128.36:6014 182.16.128.36:6003 d138 1.00 1 -30.56 139 1 1 182.16.128.36:6015 182.16.128.36:6003 d139 1.00 1 -30.56 140 1 1 182.16.128.38:6000 182.16.128.38:6003 d140 1.00 2 38.89 141 1 1 182.16.128.38:6006 182.16.128.38:6003 d141 1.00 2 38.89 142 1 1 182.16.128.38:6007 182.16.128.38:6003 d142 1.00 2 38.89 143 1 1 182.16.128.38:6008 182.16.128.38:6003 d143 1.00 2 38.89 144 1 1 182.16.128.38:6009 182.16.128.38:6003 d144 1.00 2 38.89 145 1 1 182.16.128.38:6010 182.16.128.38:6003 d145 1.00 1 -30.56 146 1 1 182.16.128.38:6011 182.16.128.38:6003 d146 1.00 1 -30.56 147 1 1 182.16.128.38:6012 182.16.128.38:6003 d147 1.00 1 -30.56 148 1 1 182.16.128.38:6013 182.16.128.38:6003 d148 1.00 1 -30.56 149 1 1 182.16.128.38:6014 182.16.128.38:6003 d149 1.00 1 -30.56 150 1 1 182.16.128.38:6015 182.16.128.38:6003 d150 1.00 1 -30.56 151 1 1 182.16.128.39:6000 182.16.128.39:6003 d151 1.00 2 38.89 152 1 1 182.16.128.39:6006 182.16.128.39:6003 d152 1.00 2 38.89 153 1 1 182.16.128.39:6007 182.16.128.39:6003 d153 1.00 2 38.89 154 1 1 182.16.128.39:6008 182.16.128.39:6003 d154 1.00 2 38.89 155 1 1 182.16.128.39:6009 182.16.128.39:6003 d155 1.00 2 38.89 156 1 1 182.16.128.39:6010 182.16.128.39:6003 d156 1.00 1 -30.56 157 1 1 182.16.128.39:6011 182.16.128.39:6003 d157 1.00 1 -30.56 158 1 1 182.16.128.39:6012 182.16.128.39:6003 d158 1.00 1 -30.56 159 1 1 182.16.128.39:6013 182.16.128.39:6003 d159 1.00 1 -30.56 160 1 1 182.16.128.39:6014 182.16.128.39:6003 d160 1.00 1 -30.56 161 1 1 182.16.128.39:6015 182.16.128.39:6003 d161 1.00 1 -30.56 55 1 1 182.16.128.5:6000 182.16.128.5:6003 d55 2000.40 2789 -3.18 56 1 1 182.16.128.5:6006 182.16.128.5:6003 d56 2000.40 2789 -3.18 57 1 1 182.16.128.5:6007 182.16.128.5:6003 d57 2000.40 2789 -3.18 58 1 1 182.16.128.5:6008 182.16.128.5:6003 d58 2000.40 2789 -3.18 59 1 1 182.16.128.5:6009 182.16.128.5:6003 d59 2000.40 2789 -3.18 60 1 1 182.16.128.5:6010 182.16.128.5:6003 d60 2000.40 2789 -3.18 61 1 1 182.16.128.5:6011 182.16.128.5:6003 d61 2000.40 2789 -3.18 62 1 1 182.16.128.5:6012 182.16.128.5:6003 d62 2000.40 2789 -3.18 63 1 1 182.16.128.5:6013 182.16.128.5:6003 d63 2000.40 2789 -3.18 64 1 1 182.16.128.5:6014 182.16.128.5:6003 d64 2000.40 2789 -3.18 65 1 1 182.16.128.5:6015 182.16.128.5:6003 d65 2000.40 2788 -3.21 44 1 1 182.16.128.6:6000 182.16.128.6:6003 d44 269.43 376 -3.09 45 1 1 182.16.128.6:6006 182.16.128.6:6003 d45 269.43 376 -3.09 46 1 1 182.16.128.6:6007 182.16.128.6:6003 d46 269.43 376 -3.09 47 1 1 182.16.128.6:6008 182.16.128.6:6003 d47 269.43 376 -3.09 48 1 1 182.16.128.6:6009 182.16.128.6:6003 d48 269.43 376 -3.09 49 1 1 182.16.128.6:6010 182.16.128.6:6003 d49 269.43 376 -3.09 50 1 1 182.16.128.6:6011 182.16.128.6:6003 d50 269.43 376 -3.09 51 1 1 182.16.128.6:6012 182.16.128.6:6003 d51 269.43 375 -3.35 52 1 1 182.16.128.6:6013 182.16.128.6:6003 d52 269.43 375 -3.35 53 1 1 182.16.128.6:6014 182.16.128.6:6003 d53 269.43 375 -3.35 54 1 1 182.16.128.6:6015 182.16.128.6:6003 d54 269.43 375 -3.35 0 1 1 182.16.128.7:6000 182.16.128.7:6003 d0 2000.40 2789 -3.18 1 1 1 182.16.128.7:6006 182.16.128.7:6003 d1 2000.40 2789 -3.18 10 1 1 182.16.128.7:6015 182.16.128.7:6003 d10 2000.40 2788 -3.21 2 1 1 182.16.128.7:6007 182.16.128.7:6003 d2 2000.40 2789 -3.18 3 1 1 182.16.128.7:6008 182.16.128.7:6003 d3 2000.40 2789 -3.18 4 1 1 182.16.128.7:6009 182.16.128.7:6003 d4 2000.40 2789 -3.18 5 1 1 182.16.128.7:6010 182.16.128.7:6003 d5 2000.40 2789 -3.18 6 1 1 182.16.128.7:6011 182.16.128.7:6003 d6 2000.40 2789 -3.18 7 1 1 182.16.128.7:6012 182.16.128.7:6003 d7 2000.40 2789 -3.18 8 1 1 182.16.128.7:6013 182.16.128.7:6003 d8 2000.40 2789 -3.18 9 1 1 182.16.128.7:6014 182.16.128.7:6003 d9 2000.40 2789 -3.18 11 2 2 11.17.128.5:6000 11.17.128.5:6003 d11 2000.40 2979 3.42 12 2 2 11.17.128.5:6006 11.17.128.5:6003 d12 2000.40 2979 3.42 13 2 2 11.17.128.5:6007 11.17.128.5:6003 d13 2000.40 2979 3.42 15 2 2 11.17.128.5:6008 11.17.128.5:6003 d15 2000.40 2979 3.42 16 2 2 11.17.128.5:6009 11.17.128.5:6003 d16 2000.40 2979 3.42 17 2 2 11.17.128.5:6010 11.17.128.5:6003 d17 2000.40 2978 3.38 18 2 2 11.17.128.5:6011 11.17.128.5:6003 d18 2000.40 2978 3.38 19 2 2 11.17.128.5:6012 11.17.128.5:6003 d19 2000.40 2978 3.38 20 2 2 11.17.128.5:6013 11.17.128.5:6003 d20 2000.40 2978 3.38 21 2 2 11.17.128.5:6014 11.17.128.5:6003 d21 2000.40 2978 3.38 77 2 2 11.17.128.5:6015 11.17.128.5:6003 d77 2000.40 2978 3.38 33 2 2 11.17.128.6:6000 11.17.128.6:6003 d33 2000.40 2979 3.42 34 2 2 11.17.128.6:6006 11.17.128.6:6003 d34 2000.40 2979 3.42 35 2 2 11.17.128.6:6007 11.17.128.6:6003 d35 2000.40 2979 3.42 36 2 2 11.17.128.6:6008 11.17.128.6:6003 d36 2000.40 2979 3.42 37 2 2 11.17.128.6:6009 11.17.128.6:6003 d37 2000.40 2978 3.38 38 2 2 11.17.128.6:6010 11.17.128.6:6003 d38 2000.40 2978 3.38 39 2 2 11.17.128.6:6011 11.17.128.6:6003 d39 2000.40 2978 3.38 40 2 2 11.17.128.6:6012 11.17.128.6:6003 d40 2000.40 2978 3.38 41 2 2 11.17.128.6:6013 11.17.128.6:6003 d41 2000.40 2978 3.38 42 2 2 11.17.128.6:6014 11.17.128.6:6003 d42 2000.40 2978 3.38 43 2 2 11.17.128.6:6015 11.17.128.6:6003 d43 2000.40 2978 3.38 162 2 2 11.17.128.71:6000 11.17.128.71:6003 d162 1.00 2 38.89 163 2 2 11.17.128.71:6006 11.17.128.71:6003 d163 1.00 2 38.89 164 2 2 11.17.128.71:6007 11.17.128.71:6003 d164 1.00 2 38.89 165 2 2 11.17.128.71:6008 11.17.128.71:6003 d165 1.00 2 38.89 166 2 2 11.17.128.71:6009 11.17.128.71:6003 d166 1.00 1 -30.56 167 2 2 11.17.128.71:6010 11.17.128.71:6003 d167 1.00 1 -30.56 168 2 2 11.17.128.71:6011 11.17.128.71:6003 d168 1.00 1 -30.56

3 2

Re: [swift]: Swiftstack Zone/DC decommission
by Muralikrishna Gutha 14 Nov '23

14 Nov '23

Clary, Thanks for the update I was unable to locate the object.builder file in the instance where we are running swiftstack. We are currently on Swift Package Version: 2.24.0.3-1.el7 [root@hostname swift]# /opt/ss/bin/swift-ring-builder object.builder | head Ring Builder file does not exist: object.builder [root@hostname swift]# locate object.builder [root@hostname swift]#pwd /etc/swift Thanks, Murali -- This message may contain confidential and/or privileged information. If you are not the addressee or authorized to receive this on behalf of the addressee you must not use, copy, disclose or take action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply email and delete this message. Thank you.

2 1

[PTL][release][stable] Yoga - final release before branch becomes unmaintained
by Elõd Illés 13 Nov '23

13 Nov '23

Hi teams, According to the new process [1] Yoga should transition to Unmaintained as soon as possible (the original deadline for Extended Maintenance transition was planned to 2023-11-02). Similarly like before, teams have the final opportunity to release from stable/yoga before the branches will be *deleted*. In order to help the teams I have generated the list of the current *open* and *unreleased* changes in stable/yoga for repositories [2] where there are relevant patches. Please do a *final* release on Yoga if needed, before stable/yoga transition to Unmaintained. Feel free to edit and extend these lists to track your team's progress! *NOTE*: teams, please focus on wrapping up your libraries first if there is any concern about the changes, in order to avoid broken (final!!) releases! [1] https://governance.openstack.org/tc/resolutions/20230724-unmaintained-branc… [2] https://etherpad.opendev.org/p/yoga-final-release-before-unmaintained [3] https://review.opendev.org/q/topic:xena-em Thanks, Előd irc: elodilles @ #openstack-release

2 1

[neutron][ptg] 2024.1/Caracal vPTG summary
by Brian Haley 13 Nov '23

13 Nov '23

Sorry for the slowness of getting this out. Here is a summary of the discussions we had over our two days of meetings, as well as other related sessions. The etherpad link can be found at [0]. Monday, October 23rd -------------------- # OS DBPerf Cross project session [1] This session was to talk about devstack performance issues that are believed to be caused by excessive querying by Neutron and Keystone. By default, when devstack "stacks", DB counter values are collected on each of the subsystems. Keystone and Neutron are outliers on how many SELECTs they issue during this process. As this is such a high-level number (4x to 10x other users), more work is required to understand what to fix, or if it's just normal behavior. For example, there is data from just a "stack" in addition to a tempest run that might be able to narrow things down here. Slawek and Miguel have agreed to work with the oslo.db team and Mike Bayer to get a better understanding of things here. Tuesday, October 24th --------------------- # Horizon For the last few cycles the Neutron team (Lajos) has been trying to deprecate usage of neutronclient. While this work is mostly complete, there is usage in Horizon that needs to be addressed. Lajos and Amotoki will be working on this. Full notes at [2] Wednesday, October 25th ----------------------- # Retrospective As is customary the team did a retrospective of the good, bad and what could be improved. In short, the Neutron team has met its goals from last cycle of completing Sqlalchemy 2.0 and RBAC work, while continuing to deliver a few new features and a lot of bug fixes. Most of the team are long time members which helps. One place we are working on is our job stability, and we continue to look at how to increase our review velocity. Our initial proposal on the latter is to have review marathons, perhaps on a monthly basis, since it has the double effect of closing bugs in the process. # EOL/EM cadence Train just went EOL. Decision was to propose Ussuri for EOL this cycle. # Bugs The total number of open bugs against Neutron has crept up since the beginning of Bobcat, from 726 to 760. I asked people to go through any bugs they own and make sure they are in the right state or should be closed. This is something we've done in previous cycles that has reduced our count by 1/3. # Gate We now have voting SLURP jobs! I.e. from Antelope to Caracal. Rodolfo has also been working on fullstack performance to try and reduce the runtime since we hit the 3 hour limit occasionally. Action item was taken by Slawek to see if there was overlap with other tests to see what could be removed. There was a question on what versions of OVS and OVN we use in our gate, and if there is a rule on when we should bump it, especially since sometimes new features require newer versions. Action item was taken by me to create a requirements doc similar to what Nova does for libvirt. # Neutronclient deprecation As discussed above, Horizon will be the focus this cycle. # OVN drivers for stadium projects There have been efforts recently to make stadium projects work with OVN, for example VPNaaS, FWaas and Taas. Main ask for for review cycles from core contributors. # Hidden SG rules in OVN Background is at [3]. Short story is that there are a minimum set of things that must be allowed for a VM to configure and use the network, like ARP, DHCP, ND, RA, RS and MLD. ML2/OVS adds hidden SG rules for these, and OVN allows some via ACLs. Our api-ref actually documents these requirements since stateless SGs needed to add them to function properly. The only service that is not enabled by default with stateless SGs is metadata. Question was regarding if we should make these rules visible in a read-only state. # S-RBAC This is a community goal and Neutron has completed the first phase and is part of phase 2 (service role), additional testing and nova/neutron token passing TBD. Phase 3 is the Project Manager role, which is not planned for this cycle. Thursday, October 26th ---------------------- # RFE reviews Discussed three active RFEs, two related to neutron-dynamic-routing, and one for OVN interconnection. Further discussion to take place in reviews and/or drivers meeting. # Metadata changes I noticed there were a number of changes to the metadata code in flight, so we went through them to try and create order out of the chaos. Basically, we will try and merge any bug fixes first since they need to be backported, followed by features and refactors. See the etherpad for more details. # OVN DB sync utility This tool is typically run after a migration or upgrade to make sure the Neutron and OVN DBs are in sync. A number of bugs have been fixed and backported, but there might be more. An action item was taken to try and make it more robust, and also make sure usage is documented. # Active-Active L3 gateway multi-homing Frode Nordahl gave an update on this OVN work that started in the Bobcat cycle, necessary to support multiple external gateways. A little over half the code is merged, plan is to complete the rest this cycle. # Nova cross-project Sean Mooney proposed that Neutron starts making certain extensions mandatory, for example, multiple port bindings, that make migration possible. Since then other features have been added that require it as well. Should we stop supporting backends that have not implemented it? We agreed this is a good step forward to what should be the minimum requirements of Neutron. In addition: - Nova will deprecate the legacy code path that doesn't use multiple bindings and log a warning, to be deleted in E release - Neutron will do the same for any similar code - Sean will provide a blueprint explaining which API extensions will be mandatory by Nova, as well as file a Neutron RFE bug - Eventually, we'll provide release notes and a ML thread for explaining all of this (we being the Nova and Neutron PTLs) Complete notes in Nova etherpad [4]. Thanks, and if you think I missed something please send updates or corrections. -Brian [0] https://etherpad.opendev.org/p/oct2023-ptg-neutron [1] https://etherpad.opendev.org/p/oct2023-ptg-os-dbperf-crossproject [2] https://etherpad.opendev.org/p/horizon-caracal-ptg [3] https://bugs.launchpad.net/neutron/+bug/1926515 [4] https://etherpad.opendev.org/p/nova-caracal-ptg#L80

2 2

[ops][cinder][kolla] Cinder + Ceph under high load can lead to timeouts and a failure state.
by Forrest Fuqua 13 Nov '23

13 Nov '23

In my setup of Cinder Zed + Ceph Quincy under very high loads there is a race condition due to a low timeout by default in Openstack Kolla in Cinder.conf of rados_connect_timeout = 5 This can lead to a duplicate volume trying to be made on ceph and cinder erroring out a VM Build in Nova. Nova: nova.exception.BuildAbortException: Build of instance a529f7d5-5a81-44f6-a1fd-93caa39b2439 aborted: Volume b476094b-075e-4dc7-b63e-5bb58a0a9229 did not finish being created even after we waited 30 seconds or 4 attempts. And its status is error. Cinder-Volume: File "/var/lib/kolla/venv/lib/python3.9/site-packages/cinder/volume/drivers/rbd.py", line 230, in __init__\n self.volume = driver.rbd.Image(rados_ioctx,\n', ' File "rbd.pyx", line 2894, in rbd.Image.__init__\n', "rbd.Timeout: [errno 110] RBD operation timeout (error opening image b'volume-b476094b-075e-4dc7-b63e-5bb58a0a9229' at snapshot None)\n"] --- File "/var/lib/kolla/venv/lib/python3.9/site-packages/eventlet/tpool.py", line 132, in execute\n six.reraise(c, e, tb)\n', ' File "/var/lib/kolla/venv/lib/python3.9/site-packages/six.py", line 719, in reraise\n raise value\n', ' File "/var/lib/kolla/venv/lib/python3.9/site-packages/eventlet/tpool.py", line 86, in tworker\n rv = meth(*args, **kwargs)\n', ' File "rbd.pyx", line 698, in rbd.RBD.clone\n', 'rbd.ImageExists: [errno 17] RBD image already exists (error creating clone)\n'] --- I can confirm that the volume does exist in Ceph now but no longer being tracked in Cinder, leading to a leftover artifact that eats up resources and a failed VM build. This seems like behavior that should be accounted for in some way.

2 2

[swift] Swiftstack Zone/DC decomission
by Muralikrishna Gutha 13 Nov '23

13 Nov '23

Certainly, here's a rewritten version: We currently utilize SwiftStack Object Storage to meet our object storage requirements, and our cluster has a Primary Zone/Secondary Zone topology. Recently, we attempted to remove/disable the Secondary Zone. However, upon doing so, we observed a concerning trend: the storage utilization in the Primary Zone rapidly increased, reaching approximately 90%. This surge was particularly notable due to the use of the 'tworeplicaperregion' policy. In an attempt to address this, we brought the Secondary Zone nodes back online, expecting Swift to redistribute the data back to these nodes and alleviate the storage strain on the Primary Zone. Despite reintegrating the Secondary Zone nodes into the cluster, we continue to witness a persistent rise in storage utilization within the Primary Zone. Interestingly, two out of the three nodes in the Primary Zone seem to bear the brunt of the data and tokens, while the third node is not receiving a proportional distribution of data. At this juncture, we are seeking guidance on potential configurations or adjustments, such as specific knobs, that could expedite the rebalance process. Our objective is to efficiently distribute the data across nodes, particularly in the Primary Zone, and optimize the overall storage utilization in our SwiftStack Object Storage cluster. Any insights or recommendations on tuning parameters for a swifter rebalance would be highly appreciated. Thanks, Murali -- This message may contain confidential and/or privileged information. If you are not the addressee or authorized to receive this on behalf of the addressee you must not use, copy, disclose or take action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply email and delete this message. Thank you.

1 0

[tc] Next meeting Tomorrow, Tuesday Nov 14, 1800 UTC
by Jay Faulkner 13 Nov '23

13 Nov '23

Hi all, The next meeting of the OpenStack Technical Committee is tomorrow, Tuesday November 14 at 1800 UTC in #openstack-tc on OFTC. The proposed agenda is: - Roll call - Follow up on tracked action items: https://meetings.opendev.org/meetings/tc/2023/tc.2023-11-07-18.00.html - Jay to sign up to rep TC at PTG OpenInfra Live recap - Gate health check - Leaderless projects (gmann) - Implementation of Unmaintained branch statuses - draft email to openstack-discuss while we figure this out: https://etherpad.opendev.org/p/early-caracal-unmaintained-transition - 2024.1 TC Tracker - https://etherpad.opendev.org/p/tc-2024.1-tracker - Open Discussion and Reviews Thanks all, Jay Faulkner OpenStack TC Chair

1 0