- openstack-discuss - lists.openstack.org

[nova] Instance migration/resize changes availability zone
by Marc Vorwerk 25 Mar '24

25 Mar '24

Dear OpenStack Community, I am reaching out for support with an issue that is specifically affecting a single instance during migration or resize operations in our OpenStack environment. I want to emphasize that this problem is isolated and does not reflect a broader issue within our cloud setup. The issue arises when attempting a resize of the instance's flavor, which only differs in RAM+CPU specification. Unexpectedly, the instance attempts to switch its availability zone from az1 to az2, which is not the intended behavior. The instance entered an error state during the resize or migration process, with a fault message indicating 'ImageNotFound', because after the availability zone change the volume cant be reached. We use seperate ceph clusters per az. To debug this issue we enabled debug logs for the nova scheduler and found that the scheduler does not filter out any node with any of our enabled filter plugins. As a quick test we disabled a couple of compute nodes and could also verify that the ComputeFilter of the nova-scheduler still just returns the full list of all nodes in the cluster. So it seems to us that for some reason the nova-scheduler just ignores all enabled filter plugins for mentioned instance. It's worth noting that other instances with the same flavor on the same compute node do not exhibit these issues, highlighting the unique nature of this problem. Furthermore we checked all relevant database tables to see if for some reason something strange is saved for this instance but it seems to us that the instance has exactly the same attributes as other instances on this node. We are seeking insights or suggestions from anyone who might have experienced similar issues or has knowledge of potential causes and solutions. What specific logs or configuration details would be helpful for us to provide to facilitate further diagnosis? We greatly appreciate any guidance or assistance the community can offer. Best regards, Marc Vorwerk + Maximilian Stinsky

5 6

[neutron] Bug Deputy Report March 18 - March 24
by Lajos Katona 25 Mar '24

25 Mar '24

Hi Neutron Team, I was the bug deputy for Neutron from 18March to 24 March, here comes my report for the week. - https://bugs.launchpad.net/neutron/+bug/2058400 ([ovn-octavia-provider] functional jobs are unstable) - Fernando Royo started to investigate the issues - https://bugs.launchpad.net/neutron/+bug/2058400/comments/2 - https://bugs.launchpad.net/neutron/+bug/2058378 ([devstack-tobiko-neutron] stable/2023.2 periodic broken with ERROR: Cannot install tox==4.13.0, tox==4.14.0 and tox==4.14.1) - https://review.opendev.org/c/x/devstack-plugin-tobiko/+/913746 fixed the failure. - https://bugs.launchpad.net/neutron/+bug/2058712 ([openvswitch-agent] Trunk extension should not be loaded if service plugin is disabled) - In progress: https://review.opendev.org/c/openstack/neutron/+/913979 - https://bugs.launchpad.net/neutron/+bug/2055419 (network autoallocation fails for non-admin user) - Incomplete: It seems something is missing for the reproduction - https://bugs.launchpad.net/neutron/+bug/2058433 (a public network with multi subnets , but based on ovn, They can't communicate with each other) - Incomplete: I try to collect some configurations, but somebody with more OVN background should check it also. Best wishes Lajos Katona (lajoskatona)

1 0

Please Help To Setup Bumblebee VDI project based on DevStack
by Matthew Swigart 24 Mar '24

24 Mar '24

Hello, I'm trying to install the VDI project on my Ubuntu 22.04 LTS Desktop VMware workstation. So I chose Bumblebee(https://github.com/NeCTAR-RC/bumblebee) and went through several steps like configuring the devstack, installing docker, and Bumblebee project. And then, I tried to create a virtual desktop from Bumblebee and I got the below error: ValueError: badly formed hexadecimal UUID string Attached error screenshot: [image: image.png] Anybody has already experienced a similar issue or does anybody have any idea to solve this problem? Then please help me. Looking forward to hearing from you. Much appreciated. Matthew

3 2

[all][elections][ptl] 2024.2 Dalmatian: PTL Election Conclusion & Results
by Ian Y. Choi 24 Mar '24

24 Mar '24

Thank you to the electorate, to all those who voted and to all candidates who put their name forward for Project Team Lead (PTL) in this election. A healthy, open process breeds trust in our decision making capability. Thank you to all those who make this process possible. Now for the results of the PTL election process, please join me in extending congratulations to the following PTLs: * Adjutant - Dale Smith (dalees) * Barbican - Grzegorz Grasza (xek) * Blazar - Pierre Riteau (priteau) * Cinder - Jon Bernard () * Cloudkitty - Rafael Weingartner () * Cyborg - alex song (songwenping) * Designate - Michael Johnson (johnsom) * Glance - Pranali Deore (pdeore) * Heat - Takashi Kajinami (tkajinam) * Horizon - Tatiana Ovchinnikova (tmazur) * Ironic - Riccardo Pittau (rpittau) * Keystone - Dave Wilde (d34dh0r53) * Kolla - Michal Nasiadka (mnasiadka) * Magnum - Jake Yip (jakeyip) * Manila - Carlos Silva (carloss) * Masakari - sam sue () * Mistral - axel vanzaghi () * Monasca - Hasan Acar () * Neutron - Brian Haley (haleyb) * Nova - Sylvain Bauza (bauzas) * Octavia - Gregory Thiemonge (gthiemonge) * OpenStack Helm - Vladimir Kozhukalov (kozhukalov) * OpenStackAnsible - Dmitriy Rabotyagov (noonedeadpunk) * OpenStackSDK - Artem Goncharov (gtema) * Puppet OpenStack - Takashi Kajinami (tkajinam) * Quality Assurance - Martin Kopec (kopecmartin) * Rally - Andrey Kurilin (andreykurilin) * Storlets - Takashi Kajinami (tkajinam) * Sunbeam - Hemanth Nakkina () * Swift - Tim Burke (timburke) * Tacker - Yasufumi Ogawa (yasufum) * Telemetry - Erno Kuvaja (jokke_) * Venus - Eric Zhang () * Vitrage - Dmitriy Rabotyagov (noonedeadpunk) * Zaqar - Hao Wang (wanghao) * Zun - Hongbin Lu () Elections: no election required. For remaining projects (without PTL candidates), TC will work on the appointment. Election process details and results are also available here: https://governance.openstack.org/election/ Thank you to all involved in the PTL election process, Ian Y. Choi, on behalf of the OpenStack Technical Election Officials

1 0

Accessing virsh in openstack wallaby | tripleo
by Swogat Pradhan 24 Mar '24

24 Mar '24

Hi, Can someone please tell me how can i access the virsh utility on the compute node on openstack wallaby? Previously we used nova_libvirt but in wallaby there are multiple containers and i am unable to use virsh utility to edit my domain.xml file. List of containers: [root@dcn01-hci-2 ~]# podman ps | grep nova 10d981158842 172.25.201.68:8787/tripleomaster/openstack-nova-libvirt:current-tripleo kolla_start 2 months ago Up 2 months ago nova_virtlogd_wrapper e91667516afc 172.25.201.68:8787/tripleomaster/openstack-nova-libvirt:current-tripleo kolla_start 2 months ago Up 2 months ago nova_virtsecretd a4b07fe0e833 172.25.201.68:8787/tripleomaster/openstack-nova-libvirt:current-tripleo kolla_start 2 months ago Up 2 months ago nova_virtnodedevd d7e1db393e9c 172.25.201.68:8787/tripleomaster/openstack-nova-libvirt:current-tripleo kolla_start 2 months ago Up 2 months ago nova_virtstoraged 5308a171793e 172.25.201.68:8787/tripleomaster/openstack-nova-libvirt:current-tripleo kolla_start 2 months ago Up 2 months ago nova_virtqemud f490a6249ba1 172.25.201.68:8787/tripleomaster/openstack-nova-libvirt:current-tripleo kolla_start 2 months ago Up 6 weeks ago nova_virtproxyd ab0441e56957 172.25.201.68:8787/tripleomaster/openstack-nova-compute:current-tripleo kolla_start 2 months ago Up 2 months ago (healthy) nova_migration_target f35fbba5b690 172.25.201.68:8787/tripleomaster/openstack-nova-compute:current-tripleo kolla_start 2 months ago Up 6 weeks ago (unhealthy) nova_compute 82bc01993106 172.25.201.68:8787/tripleomaster/openstack-nova-libvirt:current-tripleo /usr/sbin/virtlog... 2 months ago Up 2 months ago Can someone please suggest how I can i edit an instance xml file in this setup? With regards, Swogat Pradhan

2 5

Openstack trove Error launching instance
by Shameem Farhath 23 Mar '24

23 Mar '24

Hi Team, Myself Shameem, on behalf of NerveDigital i am writing this mail. We are provisioning the trove database as a service in our existing infrastructure, and we got an ERROR on launching a DB instance. We have been troubleshooting it for a while and find no hope. Please help us to find a proper resolution for the issue. Please find the attached ERROR while launching the instance. Openstack version: yoga Operating system used: Ubuntu 22.04 Guest operating system: ubuntu jammy (downloaded from opendev tarball) Configuration files: Attached ERROR from the log: Attached *Regards,* *Shameem Farhath A* *9562440781* *www.NerveDigital.net* <http://www.NerveDigital.net>

1 0

[openstack-ansible][horizon][keystone] multidomain - LDAP admins unable to see Horizon Admin Dashboard
by Janet 'Saoirse' Sullivan 22 Mar '24

22 Mar '24

Hi, I have a multi-domain setup (LDAP and Default) and while LDAP users can see the Projects and Identity tabs in the Horizon dashboard, they can't see the Admin tab, even if they are assigned the admin role. The Default domain admin user can see this dashboard just fine. Is there a policy that needs to be changed so LDAP users with the admin role can see the Admin tab in Horizon? Openstack 2023.2.

1 0

[TC][all] Recheck analysis
by Sławek Kapłoński 22 Mar '24

22 Mar '24

Hi Stackers, Some time ago I started monitoring something what we have called "bare rechecks" which are "recheck" comments without any reason given. I was also asking various teams to improve that, try to check failures of the CI jobs and put some kind of reason in the recheck comment. That went pretty well and currently most of the teams are doing pretty good job there. See stats from last 30 days: +--------------------+---------------+--------------+-------------------+ | Team | Bare rechecks | All Rechecks | Bare rechecks [%] | +--------------------+---------------+--------------+-------------------+ | OpenStackSDK | 8 | 8 | 100.0 | | OpenStack-Helm | 41 | 41 | 100.0 | | cloudkitty | 1 | 1 | 100.0 | | monasca | 6 | 6 | 100.0 | | skyline | 2 | 2 | 100.0 | | trove | 1 | 1 | 100.0 | | rally | 1 | 1 | 100.0 | | watcher | 1 | 1 | 100.0 | | mistral | 5 | 8 | 62.5 | | kuryr | 1 | 2 | 50.0 | | zaqar | 1 | 2 | 50.0 | | horizon | 6 | 15 | 40.0 | | swift | 9 | 25 | 36.0 | | manila | 5 | 14 | 35.71 | | kolla | 14 | 68 | 20.59 | | cinder | 15 | 77 | 19.48 | | ironic | 10 | 53 | 18.87 | | glance | 11 | 59 | 18.64 | | requirements | 13 | 73 | 17.81 | | Telemetry | 2 | 12 | 16.67 | | nova | 16 | 105 | 15.24 | | OpenStackAnsible | 5 | 40 | 12.5 | | magnum | 1 | 9 | 11.11 | | neutron | 13 | 118 | 11.02 | | Quality Assurance | 3 | 30 | 10.0 | | Release Management | 1 | 12 | 8.33 | | octavia | 2 | 34 | 5.88 | | keystone | 0 | 1 | 0.0 | | OpenStack Charms | 0 | 1 | 0.0 | | oslo | 0 | 8 | 0.0 | | Puppet OpenStack | 0 | 95 | 0.0 | | barbican | 0 | 2 | 0.0 | | designate | 0 | 4 | 0.0 | | heat | 0 | 7 | 0.0 | | tacker | 0 | 63 | 0.0 | | tripleo | 0 | 4 | 0.0 | +--------------------+---------------+--------------+-------------------+ Now I though that it's time for the "next step" so I spent some time looking at the reasons of the rechecks done in last 30 days. I collected those reasons with script [1] and here are some top reasons found: +----------------------------------------------------------+----------------------+ | Recheck comment | Number of occurences | +----------------------------------------------------------+----------------------+ | recheck unrelated failure | 48 | | recheck irrelevant failure | 45 | | recheck unrelated test failure | 9 | | recheck testing | 9 | | recheck infra-failure | 9 | | recheck tempest-slow-py3 | 8 | | recheck nova-next | 8 | | recheck timeout | 7 | | recheck neutron-functional-with-uwsgi | 7 | | recheck failure not related to this change | 7 | | recheck ovn-octavia-provider-functional-master unrelated | 7 | | recheck jobs running too early | 6 | | recheck dependency updated | 5 | | recheck random failure | 5 | | recheck - bifrost bug | 5 | +----------------------------------------------------------+----------------------+ Those comments as they were put in the gerrit. Of course there was much more different comments there. I just wanted to highlight here some of the most common ones. I also asked ChatGPT to group all of them in some common groups. Below are some of the main categories of the reason found by the ChatGPT: CI/Testing Issues: Timeouts in various tests (e.g., tempest tests, tox tests). Infrastructure failures impacting CI. Kernel panics or guest panics during testing. Dependency-related failures (merged or updated). Unexpected failures or intermittent issues. Dependency Management: Dependency patches merged or updated. Dependency-related failures. Release/Documentation Updates: Release note fixes or updates. Content cleanup or removal. Documentation-related failures. Infrastructure/Tooling: Infrastructure mirror issues. Tooling updates or fixes (e.g., grub2-tools, proliantutils). Issues related to mirror servers. Specific Project/Component Issues: Nova-related failures (e.g., nova-ceph-multistore). Cinder-related failures (e.g., cinder-plugin-ceph-tempest). OpenStack SDK related failures. Configuration/Setup Problems: Problems with deployment or setup (e.g., grub2-tools missing, bifrost bug). Issues with configurations (e.g., host patterns). Network/Connection Problems: DNS resolution issues. Timeout errors related to network operations. Post-Failure Analysis: Attempts to reproduce failures. Investigation into infrastructural problems. Finally some conclusion from this analysis. It seems that our most common reason of the gate instability is something called "unrelated/irrelevant failure" and also things like "infra failure". In both cases we can't really do much to try to improve that as this information is not enough. So I would like to ask all of You to try to be more precise in the recheck comments and give more detailed reasons why You recheck Your patch. If for example You have to recheck the patch due to "unrelated failure", please write there also bug number which cause failure or simply give there name of the unrelated failed test or something like that. That way when I will do similar analysis in e.g. month from now maybe it will be easier to identify most common bugs which are causing "unrelated failures" and try to prioritize somehow work on fix of that bug. [1] https://github.com/slawqo/rechecks-stats/blob/main/rechecks_stats/rechecks_… -- Slawek Kaplonski Principal Software Engineer Red Hat

8 16

[ironic]Server does not accept power operations when running OpenStack Ironic on Redfish
by Junya Noguchi (Fujitsu) 22 Mar '24

22 Mar '24

To whom it may concern. My name is Junya Noguchi. We have the following problems and questions. Please respond. 1. Problems and Questions When we use OpenStack Ironic Redfish driver to manage Redfish capable server, the server may not accept power on operations. Server may or may not power on. We want to clarify whether this is a server-specific issue or an OpenStack Ironic bug. Occurrence event ・Auto-cleaning or instance startup failed. ・Ironic does not power on via Redfish driver (see log) ・We tried increasing the power_state_change in ironic.conf to 30 minutes, but it didn't resolved. ・It does not happen when changing Redfish to IPMI 2. Server information R272-P31 (GIGABYTE Arm Server) CPU : Ampere Altra Max 2.8GHz-3.0GHz 1/128 Memory : 256GB Disk : 1TB/SSDx3/RAID1 Server firmware : MegaRac SP-X：https://www.ami.com/megarac/ ※It states that it is compatible with IPMI/Redfish Openstaack ver : Zed Hardware interface : redfish Power interface : redfish Boot interface : pxe Deploy interface : direct Network interface : flat 3. Log File : /var/log/ironic/ironic-conductor.log 2023-12-19 10:24:47.677 384940 DEBUG sushy.connector [-] HTTP request: GET "Server FQDN"/redfish/v1/Systems/Self; headers: {'OData-Version': '4.0'}; body: None; blocking: False; timeout: 60; session arguments: {}; _op /usr/lib/python3.9/site-packages/sushy/connector.py:118 2023-12-19 10:24:49.975 384940 DEBUG sushy.connector [-] HTTP response for GET "Server FQDN"/redfish/v1/Systems/Self: status code: 200 _op /usr/lib/python3.9/site-packages/sushy/connector.py:226 2023-12-19 10:24:49.976 384940 DEBUG sushy.resources.base [-] Received representation of System /redfish/v1/Systems/Self: {'_actions': {'reset': {'allowed_values': None, 'operation_apply_time_support': {'_maintenance_window_resource': {'resource_uri': '/redfish/v1/Systems/Self'}, 'maintenance_window_duration_in_seconds': 600, 'maintenance_window_start_time': None, 'mapped_supported_values': [<ApplyTime.IMMEDIATE: 'Immediate'>, <ApplyTime.AT_MAINTENANCE_WINDOW_START: 'AtMaintenanceWindowStart'>], 'supported_values': ['Immediate', 'AtMaintenanceWindowStart']}, 'target_uri': '/redfish/v1/Systems/Self/Actions/ComputerSystem.Reset'}}, '_oem_vendors': ['Gbt'], '_settings': None, 'asset_tag': 'Free form asset tag', 'bios_version': 'F31h (SCP: 2.10.20220531)', 'boot': {'allowed_values': ['None', 'Pxe', 'Floppy', 'Cd', 'Usb', 'Hdd', 'BiosSetup', 'Utilities', 'UefiShell', 'UefiTarget', 'SDCard', 'UefiHttp', 'RemoteDrive', 'UefiBootNext'], 'enabled': <BootSourceOverrideEnabled.ONCE: 'Once'>, 'mode': <BootSourceOverrideMode.UEFI: 'UEFI'>, 'target': <BootSource.PXE: 'Pxe'>}, 'description': 'System Self', 'hostname': None, 'identity': 'Self', 'indicator_led': <IndicatorLED.OFF: 'Off'>, 'links': {'oem_vendors': None}, 'maintenance_window': None, 'manufacturer': 'GIGABYTE', 'memory_summary': {'health': 'OK', 'size_gib': 256}, 'name': 'System', 'part_number': '000000000001', 'power_state': <PowerState.OFF: 'Off'>, 'serial_number': 'XXXXXXXXXXXXXX', 'sku': '01234567890123456789AB', 'status': {'health': <Health.OK: 'OK'>, 'health_rollup': <Health.CRITICAL: 'Critical'>, 'state': <State.ENABLED: 'Enabled'>}, 'system_type': <SystemType.PHYSICAL: 'Physical'>, 'uuid': '00000000-0000-4000-8000-D85ED387A8B3'} refresh /usr/lib/python3.9/site-packages/sushy/resources/base.py:689 2023-12-19 10:24:49.976 384940 ERROR oslo.service.loopingcall [-] Dynamic backoff interval looping call 'ironic.conductor.utils.node_wait_for_power_state.<locals>._wait' failed: oslo_service.loopingcall.LoopingCallTimeOut: Looping call timed out after 1490.91 seconds 2023-12-19 10:24:49.976 384940 ERROR oslo.service.loopingcall Traceback (most recent call last): 2023-12-19 10:24:49.976 384940 ERROR oslo.service.loopingcall File "/usr/lib/python3.9/site-packages/oslo_service/loopingcall.py", line 154, in _run_loop 2023-12-19 10:24:49.976 384940 ERROR oslo.service.loopingcall idle = idle_for_func(result, self._elapsed(watch)) 2023-12-19 10:24:49.976 384940 ERROR oslo.service.loopingcall File "/usr/lib/python3.9/site-packages/oslo_service/loopingcall.py", line 349, in _idle_for 2023-12-19 10:24:49.976 384940 ERROR oslo.service.loopingcall raise LoopingCallTimeOut( 2023-12-19 10:24:49.976 384940 ERROR oslo.service.loopingcall oslo_service.loopingcall.LoopingCallTimeOut: Looping call timed out after 1490.91 seconds 2023-12-19 10:24:49.976 384940 ERROR oslo.service.loopingcall 2023-12-19 10:24:49.977 384940 ERROR ironic.conductor.utils [None req-60791dc1-85c6-44c6-a3b7-a60e3242607d 4cb102fe974543b19c3e39916c7e61c1 00cfc520ffcb4533861e005a9a1e694f - - default default] Timed out after 1200 secs waiting for power on on node a357fdab-d8d5-4260-8e72-b444fa7c5cb9.: oslo_service.loopingcall.LoopingCallTimeOut: Looping call timed out after 1490.91 seconds 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils [None req-60791dc1-85c6-44c6-a3b7-a60e3242607d 4cb102fe974543b19c3e39916c7e61c1 00cfc520ffcb4533861e005a9a1e694f - - default default] Failed to prepare node a357fdab-d8d5-4260-8e72-b444fa7c5cb9 for cleaning: Failed to set node power state to power on.: ironic.common.exception.PowerStateFailure: Failed to set node power state to power on. 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils Traceback (most recent call last): 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils File "/usr/lib/python3.9/site-packages/ironic/conductor/utils.py", line 183, in node_wait_for_power_state 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils return timer.start(initial_delay=1, timeout=retry_timeout).wait() 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils File "/usr/lib/python3.9/site-packages/eventlet/event.py", line 125, in wait 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils result = hub.switch() 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils File "/usr/lib/python3.9/site-packages/eventlet/hubs/hub.py", line 313, in switch 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils return self.greenlet.switch() 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils File "/usr/lib/python3.9/site-packages/oslo_service/loopingcall.py", line 154, in _run_loop 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils idle = idle_for_func(result, self._elapsed(watch)) 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils File "/usr/lib/python3.9/site-packages/oslo_service/loopingcall.py", line 349, in _idle_for 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils raise LoopingCallTimeOut( 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils oslo_service.loopingcall.LoopingCallTimeOut: Looping call timed out after 1490.91 seconds 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils oslo_service.loopingcall.LoopingCallTimeOut: Looping call timed out after 1490.91 seconds 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils During handling of the above exception, another exception occurred: 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils Traceback (most recent call last): 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils File "/usr/lib/python3.9/site-packages/ironic/conductor/cleaning.py", line 91, in do_node_clean 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils prepare_result = task.driver.deploy.prepare_cleaning(task) 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils File "/usr/lib/python3.9/site-packages/ironic_lib/metrics.py", line 59, in wrapped 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils result = f(*args, **kwargs) 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils File "/usr/lib/python3.9/site-packages/ironic/drivers/modules/agent_base.py", line 726, in prepare_cleaning 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils result = deploy_utils.prepare_inband_cleaning( 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils File "/usr/lib/python3.9/site-packages/ironic/drivers/modules/deploy_utils.py", line 732, in prepare_inband_cleaning 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils manager_utils.node_power_action(task, states.REBOOT) 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils File "/usr/lib/python3.9/site-packages/ironic/conductor/task_manager.py", line 153, in wrapper 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils return f(*args, **kwargs) 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils File "/usr/lib/python3.9/site-packages/ironic/conductor/utils.py", line 341, in node_power_action 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils notify_utils.emit_power_set_notification( 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils self.force_reraise() 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils raise self.value 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils File "/usr/lib/python3.9/site-packages/ironic/conductor/utils.py", line 329, in node_power_action 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils task.driver.power.reboot(task, timeout=timeout) 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils File "/usr/lib/python3.9/site-packages/ironic/conductor/task_manager.py", line 153, in wrapper 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils return f(*args, **kwargs) 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils File "/usr/lib/python3.9/site-packages/ironic/drivers/modules/redfish/power.py", line 166, in reboot 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils _set_power_state(task, system, next_state, timeout=timeout) 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils File "/usr/lib/python3.9/site-packages/ironic/drivers/modules/redfish/power.py", line 68, in _set_power_state 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils cond_utils.node_wait_for_power_state(task, target_state, 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils File "/usr/lib/python3.9/site-packages/ironic/conductor/utils.py", line 189, in node_wait_for_power_state 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils raise exception.PowerStateFailure(pstate=new_state) 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils ironic.common.exception.PowerStateFailure: Failed to set node power state to power on. 2023-12-19 10:24:49.989 384940 ERROR ironic.conductor.utils 2023-12-19 10:24:49.991 384940 DEBUG ironic.conductor.utils [None req-60791dc1-85c6-44c6-a3b7-a60e3242607d 4cb102fe974543b19c3e39916c7e61c1 00cfc520ffcb4533861e005a9a1e694f - - default default] Node a357fdab-d8d5-4260-8e72-b444fa7c5cb9 is not fast-track-able because it has an error: Failed to change power state to 'power on' by 'rebooting': Failed to set node power state to power on. is_fast_track /usr/lib/python3.9/site-packages/ironic/conductor/utils.py:1146 ---------------------------------------------------------------- 富士通株式会社先端技術開発本部ソフトウェア開発統括部基盤ソフト開発部野口惇矢(Noguchi Junya) 外線：070-3052-0998 内線：72-6042-3844 mail：noguchi.junya(a)jp.fujitsu.com

3 2

[Trove][kolla] trove db instance failed
by jayesh chaudhari 22 Mar '24

22 Mar '24

Hi All, I have enabled trove in my kolla ansible yoga setup. But my trove database instance is getting stuck in build state and then eventually failed after some time with polling timeout error. I have created trove management network which is my vlan network, also my control plane and data plane traffic is segregated through different interfaces. Now when I am creating db instance in admin tenant, my openstack instance in service tenant which is getting created do not have any network. I have tested trove-guest-image manually which is working fine. Am I missing something here ? Please suggest Thanks, Jayesh

1 0

Questi
by Franck VEDEL 22 Mar '24

22 Mar '24

Good morning, I currently have an Openstack cluster made up of 3 nodes, an iscsi bay (10T), 576 G of Ram, 10T, 288vcpus. This cluster is used by around 150 students, but is reaching its limits. Having obtained a budget to set up a larger cluster, I am wondering about the choice of the number of nodes, their role (how many controllers, network, compute, etc.) and above all what solution for storage. Let's imagine a budget to buy 6 servers with good capacities, is the right choice Ceph storage (with cinder and rdb?) on the Openstack cluster nodes? Do we need 3 servers for a Ceph cluster and 3 for the Openstack part (in this case I lose capacity for the "compute" part)... I don't know what the right choices are and above all, I have a little afraid of going in the wrong directions. Could any of you guide me, or give me links to sites that could help me (and that I haven't seen). Thanks in advance Franck VEDEL Dép. Réseaux Informatiques & Télécoms IUT1 - Univ GRENOBLE Alpes 0476824462 Stages, Alternance, Emploi.

7 15

[release] Release countdown for week R-2, March 25-29
by Herve Beraud 22 Mar '24

22 Mar '24

Development Focus ----------------- We are on the final mile of the 2024.1 Caracal development cycle! Remember that the 2024.1 Caracal final release will include the latest release candidate (for cycle-with-rc deliverables) or the latest intermediary release (for cycle-with-intermediary deliverables) available. March 28 is the deadline for final 2024.1 Caracal release candidates as well as any last cycle-with-intermediary deliverables. We will then enter a quiet period until we tag the final release on April 3, 2024. Teams should be prioritizing fixing release-critical bugs, before that deadline. Otherwise it's time to start planning the 2024.2 Dalmatian development cycle, including discussing Forum and PTG sessions content, in preparation of the 2024.2 Dalmatian Virtual PTG (April 8-12, 2024).. Actions ------- Watch for any translation patches coming through on the stable/2024.1 branch and merge them quickly. If you discover a release-critical issue, please make sure to fix it on the master branch first, then backport the bugfix to the stable/2024.1 branch before triggering a new release. Also merge open release patches like https://review.opendev.org/q/topic:reno-2024.1+status:open. Please drop by #openstack-release with any questions or concerns about the upcoming release ! Upcoming Deadlines & Dates -------------------------- Final 2024.1 Caracal release: April 3, 2024 2024.2 Dalmatian Virtual PTG: April 8-12, 2024 -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/

1 0

[watcher][caracal] release critical support for new oslo.db
by James Page 22 Mar '24

22 Mar '24

Hi Watchers Ref: https://bugs.launchpad.net/watcher/+bug/2056181 Watcher 12.0.0 RC1 is not compatible with oslo.db 15.0.0 - the test suite has numerous failures as the DB api module has not been updated for the changes in oslo.db to support SQLAlchemy 2.0+. Raising awareness mainly - I can take a look at this but if anyone else with more experience of this transition can help much appreciated. Cheers James

1 0

[kolla] upgrade from 2023.1 to 2023.2
by Satish Patel 22 Mar '24

22 Mar '24

Folks, I am trying to upgrade kolla-ansible deployment from 2023.1 to 2023.2. During upgrade encounter this error TASK [service-ks-register : cinder | Granting user roles] ******************************************************************************* FAILED - RETRYING: [os-ctrl1]: cinder | Granting user roles (5 retries left). FAILED - RETRYING: [os-ctrl1]: cinder | Granting user roles (4 retries left). FAILED - RETRYING: [os-ctrl1]: cinder | Granting user roles (3 retries left). FAILED - RETRYING: [os-ctrl1]: cinder | Granting user roles (2 retries left). FAILED - RETRYING: [os-ctrl1]: cinder | Granting user roles (1 retries left). failed: [os-ctrl1] (item=cinder -> service -> service) => {"action": "os_user_role", "ansible_loop_var": "item", "attempts": 5, "changed": false, "extra_data": {"data": null, "details": null, "response": "None"}, "item": {"project": "service", "role": "service", "user": "cinder"}, "msg": "No Role found for service"} As per release notes [1] I have noticed it requires a service account but not sure what I have to add in config. [1] https://docs.openstack.org/releasenotes/kolla-ansible/2023.2.html#relnotes-…

2 3

[openstack-helm-infra][mariadb] mysql-exporter installation does not support ServiceMonitor
by mohammadk7798＠gmail.com 21 Mar '24

21 Mar '24

Hi folks. When you enable exporter monitoring, you need to create the ServiceMonitor manually and the whole "enabling monitoring" experience does not feel complete. Do we want to add support for that? I can send a pull request for this feature.

1 0

[PTG] Less than One Month Reminders!
by Kendall Nelson 20 Mar '24

20 Mar '24

Hello Everyone, The April PTG is rapidly approaching and team moderators are signing up for time to meet via the PTGbot. The schedule is filling up and we are excited to see you all! If you have not yet registered, please do so! It's free![1]. Another note about this PTG in particular, Meetpad is the default video conferencing tool. TTeam moderators are able to override the meeting URL if they choose to do so in order to make use of other tools. We do have a few zoom rooms if needed. This is the first PTG we are doing this for, so please give us feedback[2] with as many details as possible if you run into any issues with Meetpad. Open source needs open tools after all! If you have any questions, please let us know via email to ptg(a)openinfra.dev . Thank you! -Kendall Nelson Senior Upstream Developer Advocate The OpenInfra Foundation [1] http://ptg2023.openinfra.dev/ [2] https://etherpad.opendev.org/p/April2024-PTG-Feedback

2 2

[Glance] Cancelling Weekly Meeting (21st March)
by Pranali Deore 20 Mar '24

20 Mar '24

Hello, Since I won't be able to chair tomorrow's meeting and there is nothing much on agenda too, cancelling the meeting. Thanks & Regards, Pranali Deore

1 0

[swift] Can I use oslo.log / oslo.config in swift? Can we make Swift behave like the rest of OpenStack?
by Thomas Goirand 20 Mar '24

20 Mar '24

Hi, Because we had some issues with a bad rebalance in production (we didn't loose any data, but some drives went full...), I attempted to implement the Swift mitigation to prevent disk full scenarios. I found that what upstream recommends [1] is a script that's written in Python 2, and that wouldn't work at all: it's not only very old, it's also a very partial implementation. So I wrote this: https://review.opendev.org/c/openstack/swift/+/907523 But then, I was told (thanks for the review!) that I shouldn't be using oslo.config and/or oslo.log, as other parts of Swift don't do that. I do understand that the rest of Swift is like this, and therefore, the review comment is probably right. However... Back in the days, I would have understood why things would have been written that way: the Oslo libs were young, and not available everywhere, and it was kind of a good idea not to use them. But now, fast forward in 2024, I found that Swift behaving differently from the rest of OpenStack is more a problem than a feature. It's also very frustrating to use the old way with the config_parser, which can't auto-generate the config file and so on. So I am wondering: - is it still the case, that I shouldn't use oslo.config / oslo.log when adding a patch in Swift? - can we revisit this, and convert swift to oslo.config generated config files, and PBR shell-script entry points instead of binary simply dropped into the toplevel bin folder? Can we make swift smarter and behave like the rest of OpenStack? :) Cheers, Thomas Goirand (zigo) [1] https://docs.openstack.org/swift/latest/admin_guide.html#preventing-disk-fu…

2 2

[magnum] Register PTG interest
by Jake Yip 20 Mar '24

20 Mar '24

Hi all, If you are interested in attending the Magnum PTG, please fill up your interest in the planning etherpad [1] The Magnum team is very small and spread across EU and AU/NZ, so it is difficult to find slots that will suit all, but we are going to try :) P.S. There is already a tentative booking for Wed, 10 Apr, 06-08 UTC, which works for the current Cores. I am still open to shifting if there are overwhelming demand. [1] https://etherpad.opendev.org/p/apr2024-ptg-magnum-planning -- Jake Yip Technical Lead, Nectar Research Cloud

1 0

Practical workshop labs for Meetups?
by Julia Kreger 19 Mar '24

19 Mar '24

Hey folks, Yesterday, I took part in the Los Angeles OpenInfra meetup, and a question that came up from some of the organizers was if there are any practical workshop labs we can walk folks through to sort of show some practical/concrete usage aspects of OpenStack? Is anyone aware of any such materials? I ask because we are talking of trying to organize the next meetup in more a workshop style, especially as some participants did also express interest in hands on learning. Thanks in advance, -Julia

6 7

[aodh][ec2-api][sahara][trove][vitrage][watcher] autocommit was (again) removed from LegacyEngineFacade
by Takashi Kajinami 19 Mar '24

19 Mar '24

Hello, Please be aware that LegacyEngineFacade in the latest oslo db release for caracal no longer enables autocommit[1]. Removal was attempted in the previous release, as part of the overall efforts to adapt to sqlalchemy 2.0, but later reverted[2] because multiple projects need more time to adapt to the change. Now the removal is again proposed for caracal and the version currently listed in upper constraints no longer provides that functionality. Quickly searching the current usage by hound, it seems the project listed in this mail title still contains autocommit=True in LegacyEngineFacade, and is likely broken by the change. I'd suggest checking CI status to evaluate the impact. Note 1: The list may be incomplete Note 2: It seems aodh is not actually relying on autocommit and the removal might not be affecting it [1] https://review.opendev.org/c/openstack/oslo.db/+/875986 [2] https://review.opendev.org/c/openstack/oslo.db/+/895631 Thank you, Takashi -- Takashi Kajinami irc: tkajinam github: https://github.com/kajinamit launchpad: https://launchpad.net/~kajinamit

3 4

[horizon] Dalmatian Virtual PTG Schedule
by Tatiana Ovchinnikova 19 Mar '24

19 Mar '24

Hello team! Here's the etherpad for Horizon Dalmatian Virtual PTG: https://etherpad.opendev.org/p/horizon-dalmatian-ptg Feel free to add the topics you want to discuss! Our time slots for the PTG are: Tuesday, April 9, 14 UTC - 16 UTC, room havana Wednesday, April 10, 16 UTC - 17 UTC, room havana The detailed schedule is in the etherpad. Don't forget to register for PTG, if not done yet: https://openinfra.dev/ptg/ See you at the PTG! Regards, Tatiana Ovchinnikova (IRC: tmazur)

1 0

Use Openstack SDK for Swift
by Albert Braden 19 Mar '24

19 Mar '24

I'm writing a script that will use admin credentials to check user projects for containers. I'm already doing something similar for non-swift accounts, doing things like: cloud.network.networks(project_id=projectid): I read here about the Object SDK: https://docs.openstack.org/openstacksdk/latest/user/guides/object_store.html And I tried this: cloud.object_store.containers(project_id=projectid): But I get an error: Invalid query params: project_id I read the Object SDK document here and it doesn't seem to say anything about how to specify a project: https://docs.openstack.org/openstacksdk/latest/user/proxies/object_store.ht… But when I read the Swift API document, I see that I can use "/v1/{account}/{container}", so I'm guessing that there must be a way to do it via the SDK. https://docs.openstack.org/api-ref/object-store/#show-container-details-and… Is it possible to specify the project in an Object SDK call, or do I need to use the old python API for this?

2 2

[NEUTRON] Separating network with Neutron AZ
by Nguyễn Hữu Khôi 19 Mar '24

19 Mar '24

Hello. I am looking for an option to map instances to network which has same az with compute resource. It will look like [cinder]cross_az_attach = False It will be nice if we decide to run edge site Could you tell me that we will have a road map to do it? Thank you. Nguyen Huu Khoi

1 0

Tacker REST API Authentication
by Ravichandran Venkataraman (veravich) 19 Mar '24

19 Mar '24

Hi, Is it possible to use username/password for authentication of Tacker REST API calls? In the tacker.conf, I see both BASIC and OAUTH2 methods for authentication. If I set the auth_type to BASIC and configure the user_name and password values, will I be able to make the REST calls with these username and password? Thanks and regards, Ravichandran

2 3