- openstack-discuss - lists.openstack.org

[Glance][Image Encryption] Problem with data-migration in db
by Josephine Seifert 27 Aug '24

27 Aug '24

Hi, I am currently trying to implement the upgrade path for the image encryption in Glance. This should change all rows in the image_properties table where the names star with `cinder_encryption*` to `os_encrypt*` The patch is: https://review.opendev.org/c/openstack/glance/+/926905 When I try to execute the data-migration on my devstack it does not work. I wonder why? What do I have to do to make it work? Here is what I tried: ``` stack@devstack:~/devstack$ nano ../glance/glance/db/sqlalchemy/alembic_migrations/data_migrations/2024_2_migrate01_rename_encryption_metadata.py stack@devstack:~/devstack$ source /opt/stack/data/venv/bin/activate (venv) stack@devstack:~/devstack$ glance-manage db migrate 2024_2_migrate01 INFO dbcounter [-] Registered counter for database glance DEBUG oslo_db.sqlalchemy.engines [-] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=1782997) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:342}} DEBUG dbcounter [-] [1782997] Writer thread running {{(pid=1782997) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} INFO alembic.runtime.migration [-] Context impl MySQLImpl. INFO alembic.runtime.migration [-] Will assume non-transactional DDL. Database is up to date. No migrations needed. (venv) stack@devstack:~/devstack$ glance-manage db sync INFO dbcounter [-] Registered counter for database glance DEBUG oslo_db.sqlalchemy.engines [-] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=1783029) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:342}} DEBUG dbcounter [-] [1783029] Writer thread running {{(pid=1783029) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} INFO alembic.runtime.migration [-] Context impl MySQLImpl. INFO alembic.runtime.migration [-] Will assume non-transactional DDL. Database is up to date. No migrations needed. (venv) stack@devstack:~/devstack$ openstack image show enc-cirros +------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | checksum | 581caa530df8ceecfe5d4a87b847d8fa | | container_format | bare | | created_at | 2024-06-21T08:10:23Z | | disk_format | raw | | file | /v2/images/137d3237-3096-4b3f-b2c9-400819211b67/file | | id | 137d3237-3096-4b3f-b2c9-400819211b67 | | min_disk | 0 | | min_ram | 0 | | name | enc-cirros | | owner | db05e1116cb04ac689d82c738a5f103c | | properties | *cinder_encryption_key_deletion_policy*='on_image_deletion', *cinder_encryption_key_id*='cf3d7b02-c82c-4a95-b866-9a22af9f87f1', hw_rng_model='virtio', locations='[{'url': | ``` thank you Josephine (Luzi)

2 2

VM fails to start in OpenSUSE for aarch64
by Junya Noguchi (Fujitsu) 27 Aug '24

27 Aug '24

To whom it may concern I tried to use OpenSUSE aarch64 VM in OpenStack environment but failed. Do you know how to solve one of the following two problems? I look forward to your reply. 1. VM startup of OpenSUSE for OpenStack on ARM fails When I deployed a VM in an OpenStack environment using OpenSUSE 15.1 for OpenStack on ARM, the VM was successfully created but failed to start. When I checked the console log from the browser, it seems that the OS boot process is not completed. What could be the cause of this? OpenSUSE image "openSUSE-Leap-15.1-OpenStack.aarch64.qcow2" https://download.opensuse.org/repositories/Cloud:/Images:/Leap_15.2/images/… Status is Active +--------------------------------------+------------------------------+---------+ | ID | Name | Status | +--------------------------------------+------------------------------+---------+ | eb653de7-68b4-4fd8-9fe5-515d1753fd5f | OpenSUSE-Leap-Arm | ACTIVE | -------- console log -------- UEFI firmware (version edk2-20230524-4.el9_3.2 built at 00:00:00 on Feb 20 2024) Tpm2SubmitCommand - Tcg2 - Not Found Tpm2GetCapabilityPcrs fail! Tpm2SubmitCommand - Tcg2 - Not Found [2J[01;01H[=3h[2J[01;01H[2J[01;01H[=3h[2J[01;01HBdsDxe: failed to load Boot0001 "UEFI Misc Device" from PciRoot(0x0)/Pci(0x1,0x2)/Pci(0x0,0x0): Not Found BdsDxe: loading Boot0002 "EFI Internal Shell" from Fv(64074AFE-340A-4BE6-94BA-91B5B4D0F71E)/FvFile(7C04A583-9E3E-4F1C-AD65-E05268D0B4D1) BdsDxe: starting Boot0002 "EFI Internal Shell" from Fv(64074AFE-340A-4BE6-94BA-91B5B4D0F71E)/FvFile(7C04A583-9E3E-4F1C-AD65-E05268D0B4D1) [2J[01;01H[01;01HUEFI Interactive Shell v2.2 EDK II UEFI v2.70 (EDK II, 0x00010000) [1m[33m[40mMapping table[0m[37m[40m [1m[33m[40m BLK0:[0m[37m[40m [1m[37m[40mAlias(s):[0m[37m[40m PciRoot(0x0)/Pci(0x1,0x2)/Pci(0x0,0x0) [1m[33m[40m BLK1:[0m[37m[40m [1m[37m[40mAlias(s):[0m[37m[40m PciRoot(0x0)/Pci(0x1,0x2)/Pci(0x0,0x0)/HD(1,MBR,0xE6E9E590,0x800,0x13FF800) [09;01HPress [1m[37m[40mESC[0m[37m[40m in 5 seconds to skip [1m[33m[40mstartup.nsh[0m[37m[40m or any other key to continue.[09;01HPress [1m[37m[40mESC[0m[37m[40m in 4 seconds to skip [1m[33m[40mstartup.nsh[0m[37m[40m or any other key to continue.[09;01HPress [1m[37m[40mESC[0m[37m[40m in 3 seconds to skip [1m[33m[40mstartup.nsh[0m[37m[40m or any other key to continue.[09;01HPress [1m[37m[40mESC[0m[37m[40m in 2 seconds to skip [1m[33m[40mstartup.nsh[0m[37m[40m or any other key to continue.[09;01HPress [1m[37m[40mESC[0m[37m[40m in 1 seconds to skip [1m[33m[40mstartup.nsh[0m[37m[40m or any other key to continue. [10;01H[1m[33m[40mShell> [0m[37m[40m ----------------------------- 2. About Login with Non-OpenStack Images When I deployed a VM using an image without "OpenStack" in the image name, it booted successfully but I could not login. Is there a way to login to a VM created using an image like this? OpenSUSE image "openSUSE-Leap-15.2.aarch64-NoCloud" https://download.opensuse.org/repositories/Cloud:/Images:/Leap_15.2/images/… Best regards, Junya Noguchi.

2 4

[all][elections][ptl][tc] Combined PTL/TC 2025.1 cycle Election Nominations Last Days
by Sławek Kapłoński 27 Aug '24

27 Aug '24

Hi, A quick reminder that we are in the last hours for declaring PTL and TC candidacies. Nominations are open until Aug 28, 2024 23:45 UTC. If you want to stand for election, don't delay, follow the instructions at[1] to make sure the community knows your intentions. Make sure your nomination has been submitted to the openstack/election repository and approved by election officials. All members of the community are encouraged to confirm their email address and opt in to the CIVS election as described at: https://governance.openstack.org/election/#electorate Election statistics[2]: Nominations started @ 2024-08-14 23:45:00 UTC Nominations end @ 2024-08-28 23:45:00 UTC Nominations duration : 14 days, 0:00:00 Nominations remaining : 1 day, 16:27:24 Nominations progress : 87.96% --------------------------------------------------- Projects[1] : 41 Projects with candidates : 13 ( 31.71%) Projects with election : 0 ( 0.00%) --------------------------------------------------- Need election : 0 () Need appointment : 28 (Adjutant Barbican Cinder Cloudkitty Cyborg Designate Glance Heat Keystone Kuryr Magnum Manila Masakari Mistral Nova Octavia OpenStackAnsi ble OpenStackSDK Puppet_OpenStack Quality_Assurance Rally Skyline Storlets Swift Telemetry Venus Vitrage Watcher) --------------------------------------------------- TC Candidates : 3 (4 Required) =================================================== Stats gathered @ 2024-08-27 07:17:36 UTC This means that with approximately 2 days left, 28 projects will be deemed leaderless. In this case the TC will oversee PTL selection as described by [3]. Thank you, [1] https://governance.openstack.org/election/#how-to-submit-a-candidacy [2] Any open reviews at https://review.opendev.org/q/project:openstack/election+is:open+file:candid… have not been factored into these stats. [3] https://governance.openstack.org/resolutions/20141128-elections-process-for… -- Slawek Kaplonski Principal Software Engineer Red Hat

1 0

How to debug port down issue on kolla ansible
by William Muriithi 27 Aug '24

27 Aug '24

Hello I am stuck for days on network setup issues and would be grateful if you could provide any pointer on how best to proceed. We have an existed vlan based network. I would like to set up the openstack to be vlan based, that is, each project would be on a separate vlan. To achieve this, I went about doing this: - Converted the port connected to neutron_external_interface to trunk on the HP switch. I am planning to use openstack to provision VM on vlan 20 * Created network ->openstack network create --project=eng --provider-network-type=vlan --provider-physical-network=sriovtenant1 --provider-segment=20 eng --gateway 192.168.20.252 --dns-nameserver 192.168.20.1 * Created subnet -> openstack subnet create --network=eng --subnet-range=192.168.20.0/24 --allocation-pool start=192.168.20.1,end=192.168.20.220 --gateway 192.168.20.252 --dns-nameserver 192.168.20.1 eng_sub1 * Created port ->openstack port create --network eng --vnic-type=direct eng_port1 [kolla@ostack-40 tmp]$ openstack network show eng | grep provider | provider:network_type | vlan | | provider:physical_network | sriovtenant1 | | provider:segmentation_id | 20 | [kolla@ostack-40 tmp]$ The issue is however with port status. It always seems to be down: [kolla@ostack-40 tmp]$ openstack port show 511f54fb-d7f2-4e59-98db-8833bb4cf5e4 | grep -i status | data_plane_status | None | | propagate_uplink_status | None | | status | DOWN | [kolla@ostack-40 tmp]$ This is the globals.yml file we are using: --- config_owner_user: "kolla" config_owner_group: "kolla" workaround_ansible_issue_8743: yes openstack_release: "2024.1" kolla_internal_vip_address: "192.168.101.221" kolla_internal_fqdn: "example.openstack.exampletech.com" kolla_external_vip_address: "192.168.102.221" kolla_external_fqdn: "example.vm.exampletech.com" kolla_sysctl_conf_path: /etc/sysctl.d/100-kolla.conf kolla_container_engine: podman network_interface: "enp4s0f0" neutron_external_interface: "enp4s0f1" disable_firewall: "true" enable_external_api_firewalld: "false" enable_neutron_sriov: "yes" neutron_sriov_physnet_mappings: sriovtenant1: enp4s0f1 enable_neutron_dvr: "yes" openvswitch_hw_offload: "yes" enable_neutron_provider_networks: "yes" ceph_version: "reef" external_ceph_cephx_enabled: "yes" ceph_glance_user: "glance" ceph_glance_keyring: "client.{{ ceph_glance_user }}.keyring" ceph_glance_pool_name: "images" ceph_cinder_user: "cinder" ceph_cinder_keyring: "client.{{ ceph_cinder_user }}.keyring" ceph_cinder_pool_name: "volumes" ceph_cinder_backup_user: "cinder-backup" ceph_cinder_backup_keyring: "client.{{ ceph_cinder_backup_user }}.keyring" ceph_cinder_backup_pool_name: "backups" ceph_nova_keyring: "{{ ceph_cinder_keyring }}" ceph_nova_user: "{{ ceph_cinder_user }}" ceph_nova_pool_name: "vms" keystone_admin_project: "example" glance_backend_ceph: "yes" cinder_backend_ceph: "yes" nova_backend_ceph: "yes" We are certain sriov is working due to this observations: [root@ostack-40 ~]# cat /sys/class/net/enp4s0f1/device/sriov_totalvfs 63 [root@ostack-40 ~]# When I look for the string vlan, on the logs, I see this which implies that vlan seem to have been initialized fine. 2024-08-26 14:57:36.933 2 INFO neutron.plugins.ml2.drivers.type_vlan [-] Network VLAN ranges: OrderedDict([('sriovtenant1', [(9, 2000)])]) 2024-08-26 14:57:49.469 2 INFO neutron.plugins.ml2.drivers.type_vlan [None req-8326ea90-01d6-4de4-895c-5a37456e4dde - - - - - -] VlanTypeDriver initialization complete [root@ostack-40 ~]# grep -i "vlan " /var/log/kolla/neutron/neutron-server.log 2024-08-26 14:57:36.933 2 INFO neutron.plugins.ml2.drivers.type_vlan [-] Network VLAN ranges: OrderedDict([('sriovtenant1', [(9, 2000)])]) 2024-08-26 14:57:49.469 2 INFO neutron.plugins.ml2.drivers.type_vlan [None req-8326ea90-01d6-4de4-895c-5a37456e4dde - - - - - -] VlanTypeDriver initialization complete [root@ostack-40 ~]# grep -i vlan /var/log/kolla/neutron/neutron-server.log 2024-08-26 14:57:36.929 2 INFO neutron.plugins.ml2.managers [-] Configured type driver names: ['flat', 'vlan', 'vxlan'] 2024-08-26 14:57:36.933 2 INFO neutron.plugins.ml2.drivers.type_vlan [-] Network VLAN ranges: OrderedDict([('sriovtenant1', [(9, 2000)])]) 2024-08-26 14:57:36.933 2 INFO neutron.plugins.ml2.managers [-] Loaded type driver names: ['flat', 'vlan', 'vxlan'] 2024-08-26 14:57:36.934 2 INFO neutron.plugins.ml2.managers [-] Registered types: dict_keys(['flat', 'vlan', 'vxlan']) 2024-08-26 14:57:37.006 2 INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'vlan' 2024-08-26 14:57:49.469 2 INFO neutron.plugins.ml2.drivers.type_vlan [None req-8326ea90-01d6-4de4-895c-5a37456e4dde - - - - - -] VlanTypeDriver initialization complete 2024-08-26 14:57:49.494 2 INFO neutron.extensions.vlantransparent [None req-72317867-444d-42a7-8421-2ef4460e9aed - - - - - -] Disabled vlantransparent extension. 2024-08-26 14:57:49.597 2 INFO neutron.api.extensions [None req-72317867-444d-42a7-8421-2ef4460e9aed - - - - - -] Extension vlan-transparent not supported by any of loaded plugins What could we possibly be missing? Regards, William

2 1

[tc][all] OpenStack Technical Committee Weekly Summary and Meeting Agenda (2024.2/R-5)
by Goutham Pacha Ravi 26 Aug '24

26 Aug '24

Hello Stackers, Milestone-3 of the 2024.2 ("Dalmatian") release cycle is upon us this week [1]. This milestone marks several code freeze deadlines. It's the feature freeze across all OpenStack projects that follow the "cycle-with-rc" release model [2], as well as a client library code freeze and a freeze for the global requirements and constraints. Please refer to Thierry Carrez (ttx)'s email regarding the release team's priorities for this week [3]. Alongside these significant release events, we're also wrapping up the nomination period for the combined PTL/TC Elections for the 2025.1 release cycle on 2024-08-28. If you have been considering nominating yourself for any of these leadership roles, please do not delay any further [4]. I'd like to congratulate the three Outreachy interns who successfully completed their internships this week. I encourage you to read about their experiences through their blogs. Abby Nduta (NdutaRed) [5] worked with Tatiana Ovchinnikova (tmazur) on OpenStack Horizon, while Mohd Ariq (tspyderboy) [6] and Christian Okeke Chima (ccokeke) [7] worked with Carlos Eduardo da Silva (carloss), Ashley Rodriguez (ashrodri), and me on Manila UI. Outreachy interns have been a valuable part of energizing and enriching this community over the years. Their journey in this community wouldn't be possible without generous sponsorship. Currently, the OpenStack community doesn't have funding to secure an intern for Outreachy Internships running between December 2024 and May 2025. If you're able to support this program, or know someone who can, please get in touch with the OpenStack Outreachy coordinators Mahati Chamarthy (mahatic) and Dmitri Tantsur (dtantsur) off-list. You can also ping us on OFTC's #openstack-outreachy channel. === Weekly Meeting === The last weekly meeting of the OpenStack Technical Committee was held on 2024-08-20. We focused most of this meeting on the implications of testing/shipping non-OSI licensed software within OpenStack. There was consensus that the current TC guidance regarding shipping binary artifacts [8] needs to be extended. A suggestion was made to disallow bundling non-OSI-compatible licensed software within binaries built, tested, and/or shipped/hosted on community infrastructure. We also discussed the need for a policy around reacting to integrated software whose licensing changes to a non-OSI-compatible license over time. This discussion gained prominence due to the Kolla project looking to ship a container image for the Masakari service that recently began supporting the use of HashiCorp's Consul [9]. Consul is one of several HashiCorp "open source" projects licensed under a "Business Source License" [10]. This was a very interesting discussion that involved a lot of questions; I recommend reading through the meeting log for the complete context [11]. Further policy discussion was tabled for an upcoming forum/PTG session with a wider audience. The next meeting of the OpenStack Technical Committee is on 2024-08-27 at 1800 UTC in OFTC's #openstack-tc channel. The agenda is on our meeting wiki [12]. Please consider joining us. Remember, you can always propose topics for this meeting; feel free to edit the wiki [12], but be sure to leave your IRC nickname so we can call on you. === Governance Proposals === ==== Merged ==== - Inactive state extensions: Monasca | https://review.opendev.org/c/openstack/governance/+/923919 - Update DPL model reset steps | https://review.opendev.org/c/openstack/governance/+/925926 - Opt-in the oslo DPL model for the 2025.1 cycle | https://review.opendev.org/c/openstack/governance/+/926154 ==== Open for Review ==== - Charms: Retire unmaintained charms | https://review.opendev.org/c/openstack/governance/+/926793 - Retire all single charm repositories | https://review.opendev.org/c/openstack/governance/+/903490 - Define testing runtime for the 2025.1 release | https://review.opendev.org/c/openstack/governance/+/926150 === Upcoming Events === - 2024-08-28: Nomination period closes for 2025.1 OpenStack PTL+TC elections - 2024-08-29: Dalmatian-3 milestone - 2024-09-03: OpenInfra Summit Asia, Suwon - 2024-09-06: OpenInfra Days, Tokyo - 2024-09-10: OpenInfra Board Meeting: https://board.openinfra.dev/ Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] Release Schedule: https://releases.openstack.org/dalmatian/schedule.html [2] Cycle-with-RC release model: https://releases.openstack.org/reference/release_models.html#cycle-with-rc [3] Release countdown for week R-5: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [4] How to submit a candidacy: https://governance.openstack.org/election/#how-to-submit-a-candidacy [5] Abby Nduta's Outreachy Blog: https://dev.to/ndutared/ [6] Mohd Ariq's Outreachy Blog: https://medium.com/@tspyderboy [7] Christian Okeke Chima's Outreachy Blog: https://dev.to/ccokeke/ [8] TC Guidelines for Managing Releases of Binary Artifacts: https://governance.openstack.org/tc/resolutions/20170530-binary-artifacts.h… [9] License question - OpenStack Masakari: https://lists.openstack.org/archives/list/legal-discuss@lists.openstack.org… [10] HashiCorp licensing: https://www.hashicorp.com/license-faq [11] TC Meeting IRC Log, 2024-08-20: https://meetings.opendev.org/meetings/tc/2024/tc.2024-08-20-18.00.log.html [12] TC Meeting Agenda, 2024-08-27: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting

1 0

[all] Please remove sphinxcontrib-*diag from your (docs) dependencies
by Stephen Finucane 26 Aug '24

26 Aug '24

o/ A large number of projects are using the sphinxcontrib-blockdiag, sphinxcontrib- nwdiag and sphinxcontrib-seqdiag projects in their documentation. These packages are no longer maintained and have not seen releases in over 3 years. They are also not compatible with Pillow 10.x and are thus preventing us from bumping Pillow to a version that has pre-built Python 3.12 wheels [1][2]. Please remove these dependencies. In Nova, we decided to just take the build assets (SVG or PNG) and use these directly in our documentation [3]. Absent a better idea, this approach will work for you also. As someone with Python 3.12 (and soon, 3.13) as their default Python version, I would reallyu like to get Pillow bumped early in E, so by doing this work now we avoid broken docs CI jobs when that bump happens. Cheers, Stephen [1] https://review.opendev.org/c/openstack/requirements/+/927102 [2] https://bugs.launchpad.net/ironic/+bug/2026345 [3] https://review.opendev.org/c/openstack/nova/+/906309

1 0

[telemetry] [kolla] help measuring neutron router bandwidth
by garcetto 26 Aug '24

26 Aug '24

good afternoon, i would like to get bandwidth usage of neutron router interfaces, using kolla-ansible, enabled ceilometer and gnocchi, but if i try to enumerate with: openstack metric resource list --max-width 1000 under type i dont see BANDWIDTH as indicated in docs ( https://docs.openstack.org/ceilometer/zed/admin/telemetry-measurements.html… ) [image: image.png] what am i missing? thank you.

1 0

[neutron][eventlet] RPC handler thread model incompatible with eventlet
by Rodolfo Alonso Hernandez 26 Aug '24

26 Aug '24

Hello Neutrinos: I think you are aware about the effort to move all CI jobs to use WSGI. Other OpenStack projects have already migrated to this implementation to the detriment of the eventlet server. That will help us in the evenlet deprecation task. However some modules are still using eventlet, thus even when using WSGI in the Neutron API, it is still needed to monkey patch the system libraries. The most recent case is LP#2075147 [1]. To that purpose [2] was proposed. Again, we had another adverse effect of [2] in the RPC event handler, that is used to inform the agents, via RPC, about resource events (creation, update, deletion); this is why only the ML2/OVS jobs were affected in the CI, as opposed to ML2/OVN that is not using RPC. With [2] in place, the RPC handler thread model was not working correctly (check LP#2077790 [3]). The thread in charge of sending the RPC messages got stuck during the DB retrieval process or the RPC cast itself. That was leading to a delay in the event processing in the agents. In the CI, the most affected tests were those related to the SG rule creation/deletion. The OVS agent didn't receive on time the SG rule creation/deletion and didn't create the needed OF rules in the local OVS. In order to fix this issue, [4] has been proposed. This patch proposes to handle the RPC messages in the same thread of the API call. Because the RPC call is a ``cast`` call (no reply is expected), that won't block the API call and will just increase the time a bit. Once eventlet is deprecated and removed, it would be possible to go back to the previous model using a thread (preemptive in this case) to make the RPC calls. Regards. [1]https://bugs.launchpad.net/neutron/+bug/2075147 [2]https://review.opendev.org/c/openstack/neutron/+/925376 [3]https://bugs.launchpad.net/neutron/+bug/2077790 [4]https://review.opendev.org/c/openstack/neutron/+/926922

1 1

[neutron] bug deputy report for week of 2024-08-19
by Bence Romsics 26 Aug '24

26 Aug '24

Hi All, We had the following new bugs last week: OVN-Octavia (not yet triaged): * https://bugs.launchpad.net/neutron/+bug/2077399 OVN-Octavia maintainers please have a look! High: * https://bugs.launchpad.net/neutron/+bug/2077790 [eventlet] RPC handler thread model is incompatible with eventlet fix proposed: https://review.opendev.org/c/openstack/neutron/+/926922 Medium: * https://bugs.launchpad.net/neutron/+bug/2077532 Two floatingip_agent_gateways are created which are bound to the same agent. assigned to Anton Low: * https://bugs.launchpad.net/neutron/+bug/2077351 "Error formatting log line" sometimes seen in l3-agent log fix proposed: https://review.opendev.org/c/openstack/neutron/+/926566 Incomplete: * https://bugs.launchpad.net/neutron/+bug/2077533 An error in processing one DVR router can lead to connectivity issues for other routers discussing with Anton whether a standalone bug or a repercussion of #2077532, will follow up RFE: * https://bugs.launchpad.net/neutron/+bug/2077596 [rfe][fwaas] Add normalized_cidr column to firewall rules Cheers, Bence

1 0

[release] [ironic] [swift] Missing releases
by Thierry Carrez 26 Aug '24

26 Aug '24

Quick reminder that we'll need a release very soon for a number of deliverables following a cycle-with-intermediary release model but which have not done *any* release yet in the Dalmatian cycle: bifrost ironic-prometheus-exporter ironic-python-agent-builder ironic-ui networking-baremetal networking-generic-switch swift Those should be released ASAP, and in all cases before September 13th, 2024, so that we have a release to include in the final 2024.2 release. -- Thierry Carrez (ttx)

2 1

[release] Release countdown for week R-5
by Thierry Carrez 23 Aug '24

23 Aug '24

Development Focus ----------------- We are getting close to the end of the 2024.2 "Dalmatian" cycle! Next week on August 29th, 2024 is the dalmatian-3 milestone, also known as feature freeze. It's time to wrap up feature work in the services and their client libraries, and defer features that won't make it to the 2025.1 "Epoxy" cycle. General Information ------------------- This coming week is the deadline for client libraries: their last feature release needs to happen before "Client library freeze" on August 29th, 2024. Only bugfix releases will be allowed beyond this point. When requesting those library releases, you can also include the stable/2024.2 branching request with the review. As an example, see the "branches" section here: https://opendev.org/openstack/releases/src/branch/master/deliverables/pike/… Dalmatian-3 is also the deadline for feature work in all OpenStack deliverables following the cycle-with-rc model. To help those projects produce a first release candidate in time, only bugfixes should be allowed in the master branch beyond this point. Any feature work past that deadline has to be raised as a Feature Freeze Exception (FFE) and approved by the team PTL. Finally, feature freeze is also a good time for submitting a first version of your cycle-highlights. Cycle highlights are the raw data that helps shape what is communicated in press releases and other release activity at the end of the cycle, avoiding direct contacts from marketing folks. See https://docs.openstack.org/project-team-guide/release-management.html#cycle… for more details. Upcoming Deadlines & Dates -------------------------- Dalmatian-3 milestone (feature freeze): August 29th, 2024 (R-5 week) Cycle Highlights Due: September 5th, 2024 (R-4 week) RC1 deadline: September 13th, 2024 (R-3 week) Final RC deadline: September 26th, 2024 (R-1 week) Final 2024.2 Dalmatian release: October 2nd, 2024 OpenInfra Summit Asia: September 3-4, 2024 -- Thierry Carrez (ttx)

1 0

Openstack Instance Snapshot and Rescue Problem
by meteccinar＠gmail.com 23 Aug '24

23 Aug '24

Hi everyone, We use Ceph in Cinder as Block Storage. Firsly, we create a VM with volume (volume create automatically) and we write a data in the VM and take an snapshot after wrote data. Then we write data in main VM again and take second snapshot. But, when we try to rescue instance related main VM with first snapshot, there was an error occured. Nova Log: 2024-08-22 08:23:25.594 19 WARNING nova.compute.api [None req-eb3b4856-c8be-4926-be51-99aaec4de0e0 41040b4a21e34904b6906c12b228d8e9 67ddd9f771be452a808ba3458af544f8 - - default default] Unable to rescue an instance using a volume snapshot image with img_block_device_mapping image properties set Horizon Output: Requested rescue image 'ddc2df28-c4f5-434b-9f9a-030e56425fc5' is not supported (HTTP 400) (Request-ID: req-bd8037a1-3cd4-43fe-a945-e809b1f8ce16) Ceph Block: https://imgtr.ee/images/2024/08/22/2108d8aacee12eaf800377593a229659.png Instance Snapshot Image: https://imgtr.ee/images/2024/08/22/ca1fa81683acc0518ac161e7f97fda0e.jpeg Could it be any configuration wrong ? We will wait for your answers. Thanks.

5 8

[nova][OpenStackSDK][vitrage][zaqar] broken gates in libraries
by Elõd Illés 22 Aug '24

22 Aug '24

Hi teams in $subject, Since we are close to Feature Freeze period and Dalmatian-3 milestone, the relmgt team checked the status of the gates of libraries [1] and the following repositories seem to have broken gates on their master branches: python-novaclient osc-lib python-vitrageclient python-zaqarclient Could the teams check the broken jobs please and fix them as soon as possible? [1] https://review.opendev.org/q/topic:release-health-check-cwi-dalmatian+is:op… Thanks in advance, Előd Illés irc: elodilles @ #openstack-release

1 0

[neutron] dsvm-fullstack test suite: issues with ovs datapaths when running locally?
by Ihar Hrachyshka 22 Aug '24

22 Aug '24

Hi all, This must be simple to do, but when I try to follow the guide in a vagrant box to run the fullstack suite locally, I am hitting a wall. I suspect I miss something obvious but I can't figure it out. I'm following this doc: https://docs.openstack.org/neutron/latest/contributor/testing/fullstack.htm… I start with an empty ubuntu 22.04 vagrant box with enough memory, disk, cpu. Then I check out the neutron repo and devstack into my home dir. I also `mkdir -p /opt/stack/ && chmod a+rwx /opt/stack` and clone `requirements` repo in /opt/stack/requirements. (Otherwise configure_for_func_testing script fails.) Then I execute: export VENV=dsvm-fullstack tools/configure_for_func_testing.sh ~/devstack -i (I am not sure setting VENV envvar is needed, but that's what is in the doc) It successfully completes (I see "Phew, we're done!" message from the script.) I then: $ tox -e dsvm-fullstack neutron.tests.fullstack.test_logging (or any other test case that triggers Open vSwitch) and the test just fails with timeout. And when I look into logs, I see interface additions failing and in vswitchd log: Aug 21 20:24:26 ubuntu ovs-vswitchd[42747]: ovs|00032|bridge|ERR|failed to create bridge br-eth68909740e: No such file or directory Aug 21 20:24:26 ubuntu ovs-vswitchd[42747]: ovs|00031|ofproto|ERR|failed to open datapath br-eth68909740e: No such file or directory Aug 21 20:24:26 ubuntu ovs-vswitchd[42747]: ovs|00030|ofproto_dpif|ERR|failed to open datapath of type system: No such file or directory (there are more errors of the same...) I disabled apparmor as the doc suggests. I also confirmed that the `openvswitch` kernel module is loaded (it is not loaded after ovs and ovn are built from source as part of configure_for_func_testing.sh execution). I also tried to run the suite as root. (I don't think this is needed, but I figured I'll check.) Anyone else running into this? How do you guys execute the suite locally? Ihar

2 2

[all][elections][ptl][tc] Combined PTL/TC 2025.1 cycle Election Nominations Kickoff
by Slawomir Kaplonski 22 Aug '24

22 Aug '24

Nominations for OpenStack PTLs (Project Team Leads) and TC (Technical Committee) positions (4 positions) are now open and will remain open until Aug 28, 2024 23:45 UTC. All nominations must be submitted as a text file to the openstack/election repository as explained at https://governance.openstack.org/election/#how-to-submit-a-candidacy Please make sure to follow the candidacy file naming convention: candidates/2025.1/<Project_Name or "TC">/<email_address> (for example, "candidates/2025.1/TC/stacker(a)example.org"). The name of the file should match an email address for your current OpenStack Foundation Individual Membership. Take this opportunity to ensure that your OSF member profile contains current information: https://www.openstack.org/profile/ Any OpenStack Foundation Individual Member can propose their candidacy for an available, directly-elected seat on the Technical Committee. In order to be an eligible candidate for PTL you must be an OpenStack Foundation Individual Member. PTL candidates must also have contributed to the corresponding team during the 2024.1 to 2024.2 timeframe, Sep 15, 2023 00:00 UTC - Aug 28, 2024 00:00 UTC. Your Gerrit account must also have a verified email address matching the one used in your candidacy filename. Both PTL and TC elections will be held from Sep 04, 2024 23:45 UTC through to Sep 18, 2024 23:45 UTC. The electorate for the TC election are the OpenStack Foundation Individual Members who have a code contribution to one of the official teams over the 2024.1 to 2024.2 timeframe, Sep 15, 2023 00:00 UTC - Aug 28, 2024 00:00 UTC, as well as any Extra ACs who are acknowledged by the TC. The electorate for a PTL election are the OpenStack Foundation Individual Members who have a code contribution over the 2024.1 to 2024.2 timeframe, Sep 15, 2023 00:00 UTC - Aug 28, 2024 00:00 UTC, in a deliverable repository maintained by the team which the PTL would lead, as well as the Extra ACs who are acknowledged by the TC for that specific team. The list of project teams can be found at https://governance.openstack.org/tc/reference/projects/ and their individual team pages include lists of corresponding Extra ACs. Please find below the timeline: PTL + TC nomination starts @ Aug 14, 2024 23:45 UTC PTL + TC nomination ends @ Aug 28, 2024 23:45 UTC TC campaigning starts @ Aug 28, 2024 23:45 UTC TC campaigning ends @ Sep 04, 2024 23:45 UTC PTL + TC elections start @ Sep 04, 2024 23:45 UTC PTL + TC elections end @ Sep 18, 2024 23:45 UTC Shortly after election officials approve candidates, they will be listed on the https://governance.openstack.org/election/ page. The electorate is requested to confirm their email addresses in Gerrit prior to 2024-08-28 00:00:00+00:00, so that the emailed ballots are sent to the correct email address. This email address should match one which was provided in your foundation member profile as well. Gerrit account information and OSF member profiles can be updated at https://review.openstack.org/#/settings/contact and https://www.openstack.org/profile/ accordingly. If you have any questions please be sure to either ask them on the mailing list or to the elections officials: https://governance.openstack.org/election/#election-officials -- Best regards Slawek Kaplonski skaplons(a)redhat.com

5 5

[all][elections][ptl][tc] Combined PTL/TC 2024.2 cycle Election Nominations progress
by Sławek Kapłoński 22 Aug '24

22 Aug '24

Hi, Here is report about PTL/TC 2024.2 cycle nomination progress so far. If you consider to nominate yourself for the PTL or TC role, please do so in next 6 days. Nominations started @ 2024-08-14 23:45:00 UTC Nominations end @ 2024-08-28 23:45:00 UTC Nominations duration : 14 days, 0:00:00 Nominations remaining : 6 days, 11:07:57 Nominations progress : 53.83% --------------------------------------------------- Projects[1] : 41 Projects with candidates : 8 ( 19.51%) Projects with election : 0 ( 0.00%) --------------------------------------------------- Need election : 0 () Need appointment : 33 (Adjutant Barbican Cinder Cloudkitty Cyborg Designate Glance Heat Keystone Kury r Magnum Manila Masakari Mistral Monasca Neutron Nova Octavia OpenStackAnsible OpenStackSDK Puppet_OpenStack Q uality_Assurance Rally Skyline Storlets Swift Tacker Telemetry Trove Venus Vitrage Watcher Zun) --------------------------------------------------- TC Candidates : 3 (4 Required) =================================================== Stats gathered @ 2024-08-22 12:37:03 UTC -- Slawek Kaplonski Principal Software Engineer Red Hat

1 0

Troubleshooting ERROR status after starting instance from Ubuntu Image
by klaus.becker＠webmen.de 22 Aug '24

22 Aug '24

Hi OpenStackers! After a vacation I continue to get my first OpenStack work (one node guided install). What works: I managed to install OpenStack on Ubuntu 22.04.4 LTS on an older DELL server with plenty of RAM and Disks I used Snap Install I can use horizon to create my VMs Problem: After I made a few instances from the Ubuntu templates with different flavors, I am Getting the status „ERROR“ for each of these instances. Trying to shutdown an instance I am getting an Error: Questions: What is the best practice for troubleshooting regarding my installation type (single node, snap)? Where do I find log-files? Google brings different solutions, but the paths and files do not exist for my installation? I tried a few suggestions: Openstack instance generating status error – Fix it now <https://bobcares.com/blog/openstack-instance-generating-status-error/> ( /var/lib/nova/ and /var/lib/nova and /var/log/libvirt/qemu do not exist ) I found: ~snap/juju ~snap/openstack ~/snap/openstack$ ls -ahl total 20K drwxr-xr-x 5 webmen webmen 4.0K Aug 13 10:54 . drwx------ 4 webmen webmen 4.0K Jul 24 13:59 .. drwxr-xr-x 4 webmen webmen 4.0K Jul 19 12:53 576 drwxr-xr-x 4 webmen webmen 4.0K Jul 19 12:53 577 drwxr-xr-x 4 webmen webmen 4.0K Jul 17 12:16 common lrwxrwxrwx 1 webmen webmen 3 Jul 24 10:04 current -> 560 The link "current —> 560 has no target here! In „common", there are sunbeam-logs Maybe there is a misconfiguration with the soft-network. I had a hard time understanding the different networks I need to access VMs from the LAN. Finaly I found then logs using "find -name nova“ /var/snap/openstack-hypervisor/common/log webmen@ada:/var/snap/openstack-hypervisor/common/log$ ls -ahl total 92K drwxr-xr-x 7 root root 4.0K Jul 17 12:39 . drwxr-xr-x 8 root root 4.0K Jul 24 12:54 .. drwxr-xr-x 3 root root 4.0K Jul 17 12:39 libvirt drwxr-xr-x 2 root root 4.0K Jul 17 12:39 neutron -rw-r--r-- 1 root root 58K Aug 16 13:06 neutron.log drwxr-xr-x 2 root root 4.0K Jul 17 12:39 nova drwxr-xr-x 2 root root 4.0K Jul 17 12:39 openvswitch drwxr-xr-x 2 root root 4.0K Jul 17 12:39 ovn webmen@ada:/var/snap/openstack-hypervisor/common/log$ But most dirs are empty webmen@ada:/var/snap/openstack-hypervisor/common/log$ tree . ├── libvirt │ └── qemu ├── neutron ├── neutron.log ├── nova ├── openvswitch │ ├── ovsdb-server.log │ └── ovs-vswitchd.log └── ovn └── ovn-controller.log But I did not find the logs of glance webmen@ada:/$ sudo find -name glance ./var/snap/microk8s/common/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/995/fs/var/lib/glance ./var/snap/microk8s/common/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/995/fs/etc/glance ./var/snap/microk8s/common/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1192/fs/var/lib/glance ./var/snap/microk8s/common/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1192/fs/etc/glance ./var/snap/microk8s/common/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/915/fs/usr/bin/glance ./var/snap/microk8s/common/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/894/fs/usr/bin/glance ./var/snap/microk8s/common/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/891/fs/usr/bin/glance ./var/snap/microk8s/common/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/205/fs/usr/bin/glance ./var/snap/microk8s/common/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/234/fs/var/log/glance ./var/snap/microk8s/common/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/234/fs/var/lib/glance ./var/snap/microk8s/common/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/234/fs/usr/share/doc/glance ./var/snap/microk8s/common/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/234/fs/usr/bin/glance ./var/snap/microk8s/common/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/234/fs/usr/lib/python3/dist-packages/glance ./var/snap/microk8s/common/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/234/fs/etc/glance ./snap/openstack-hypervisor/153/usr/bin/glance ./snap/openstack/577/bin/glance ./snap/openstack/577/lib/python3.10/site-packages/muranoclient/glance ./snap/openstack/576/bin/glance ./snap/openstack/576/lib/python3.10/site-packages/muranoclient/glance find: ‘./proc/3340174’: No such file or directory find: ‘./proc/3361710’: No such file or directory find: ‘./proc/3361712’: No such file or directory find: ‘./proc/3361763’: No such file or directory find: ‘./proc/3364581’: No such file or directory find: ‘./proc/3364766’: No such file or directory find: ‘./proc/3364791’: No such file or directory htop: server lose very high The server still has a load average of 4.68 running for 4 days by now. The hungry processes are: 5 processes of /charm/bin/pebble run —create-dies —hold —http :38813 —verbose /snap/micro8s/7038/kubelite/… Here a copy of top (htop did not let me copy the list) PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1272 root 20 0 11.4g 238528 59048 S 83.3 0.1 3596:35 containerd 21051 root 20 0 1638280 260508 7088 S 72.5 0.1 266:36.35 pebble 21945 root 20 0 1305356 52048 7088 S 71.8 0.0 91:01.52 pebble 20543 root 20 0 1704412 220848 7088 S 69.5 0.1 265:03.19 pebble 2994 root 20 0 1854984 606616 104600 S 37.0 0.3 2957:41 kubelite 1275 root 20 0 3586472 340048 19788 S 23.6 0.2 1822:17 k8s-dqlite 3340174 root 20 0 2135364 31952 19516 S 6.2 0.0 0:00.19 alertmanager 1749555 snap_da+ 20 0 3242712 924312 42656 S 5.9 0.5 67:37.06 mysqld 78926 snap_da+ 20 0 3490924 1.1g 42428 S 5.6 0.6 91:53.33 mysqld 1267 root 20 0 4276408 2.3g 55224 S 5.2 1.2 529:17.09 mongod So… how do I get rid of that status ERROR in Horizon? Best regards Klaus Becker Webmen Internet GmbH Faulenstraße 12 28195 Bremen +49 421 24 34 94-0 info(a)webmen.de www.webmen.de Geschäftsführerin: Christiane Niebuhr-Redder Prokuristin: Kathleen Marx-Colonius Amtsgericht Bremen HRB 16685 USt-ID DE 179413676 Facebook Instagram LinkedIn

2 2

[glance][cinder] No format check on glance image uploading vs. thorough image format checks on cinder
by Florent Le Lain 21 Aug '24

21 Aug '24

Hello folks, Since resolution of vmdk related CVE<https://bugs.launchpad.net/nova/+bug/1996188>, Cinder performs extra checks on format of images used for specific actions, like creating a volume out of an image for instance. See related cinder commit: https://review.opendev.org/c/openstack/cinder/+/871625 Those new checks first led to failures on our tempest tests, on tests that were passing before. E.g.: tempest.api.compute.servers.test_create_server.ServersTestBootFromVolume It turned out that tests were failing because of a discrepancy between actual format of the image (qcow2 here) and declared format in glance db (raw). To us, this raises the question of format checks responsibility throughout images lifecycle, between Glance and Cinder. Today, afaik, Glance will: - tag as 'raw' any image uploaded without any format specified - tag the image with the format provided by the customer, whatever the actual format of the image is whereas Cinder is now more picky, refusing to deal with an image that claims to be of X format but is actually Y. If you do confirm these observations, in your opinion, what would be the best option among: 1. leave it as is, after all openstack customer is responsible for the accuracy of the information provided on image upload and should not be surprised by downstream mismatches when using the image 2. strengthen customer's responsibility of bullet 1. by making image format declaration mandatory on image upload 3. enforce format checks on glance on image upload 4. other way you may think of? something upstream I may have missed? Thank you for your opinion! Regards, Florent Le Lain SRE Team Lead – Public Cloud OVHcloud Rennes [1672217009737]

2 1

Openstack Backup and Restore
by meteccinar＠gmail.com 21 Aug '24

21 Aug '24

Hi everyone, we make cloud region for our company. Using Openstack with Ceph (as Block Storage). We configured backup driver field as BackupDriverCeph in the cinder.conf file. Also created volume and backups pools in Ceph Rados. We have a few problems/questions. We have developed an API that regularly backs up VMs (instances) that work with volumes. The process works as follows: First, we use the create_server_image() method from the OpenStack API to create an image of the instance. This results in an image of the instance being automatically created in OpenStack with a Snapshot type (including metadata such as direct_url, stores, schema, etc.). Simultaneously, a snapshot is also automatically created using with the create_server() command. We then take backups of all volumes attached to the instance. In our restore API, we use the image created from the instance backup mentioned above. We change block_device_mapping field and add the IDs of the volume backups to the block_device_mapping field of the image and use the create_server(image_name, volume_from_backed_volume) method in the OpenStack API to create a new instance. Then new volumes are created from volume backups and attach new volumes in new restored instance. The problem we're encountering is that when using the create_server command, the image created from the instance backup is tied to the direct_url, which points to the RBD URL of the snapshot. As a result, the instance does not running with new volume. Note: If we create a new VM with new volume from volume backup in Horizon. Instance running successfully. What steps should we take to resolve this? Is there a flaw in our approach when designing the API? What is the best practice we should follow when creating an API for backup and restore (especially for instances that boot from volumes)? Is there a configuration we need to adjust in Ceph? We would appreciate your advice on the best practices for backup and restore, particularly for instances that boot from volumes. Kind regards,

3 6

[neutron] Moving CI Meeting to Monday 1400 UTC
by Yatin Karel 21 Aug '24

21 Aug '24

Hello Neutron Team !! As discussed/agreed over the last couple of CI meetings, pushed the changes[1][2] for the new timing. [1] https://review.opendev.org/c/opendev/irc-meetings/+/926763 [2] https://etherpad.opendev.org/p/neutron-ci-meetings -- Thanks and Regards Yatin Karel

1 0

Magnum Caracal: cluster creation goes through state CREATION_FAILED before CREATION_COMPLETE
by Michel Jouvin 21 Aug '24

21 Aug '24

Hi, I continue my tests with Magnum Caracal and found another strange behaviour during cluster creation. It goes through state CREATION_FAILED before reaching the state CREATION_COMPLETE (HEALTHY). It is very confusing. Looking at Heat engine logs, it happens during load balancer creation, at the time the LB amphora has been created and Octavia is waiting to be able to connect it (typically one minue). During this period, Heat logs the following error: ----------- 2024-08-21 09:54:52.194 3200318 INFO heat.engine.resource [None req-610b24bd-7511-4d31-9f8c-b922162a7648 jouvin - - - - -] CREATE: LoadBalancer "loadbalancer" [670985ef-5808-4226-ad1b-b1244871de16] Stack "test-mj-1-30-m1n1-bis-ipufjomqmnnf-etcd_lb-d5fuqygvpp2j" [5c5905bf-f538-4a42-93ce-38d96b7c6603] 2024-08-21 09:54:52.194 3200318 ERROR heat.engine.resource Traceback (most recent call last): 2024-08-21 09:54:52.194 3200318 ERROR heat.engine.resource File "/usr/lib/python3.9/site-packages/heat/engine/resource.py", line 922, in _action_recorder 2024-08-21 09:54:52.194 3200318 ERROR heat.engine.resource yield 2024-08-21 09:54:52.194 3200318 ERROR heat.engine.resource File "/usr/lib/python3.9/site-packages/heat/engine/resource.py", line 1034, in _do_action 2024-08-21 09:54:52.194 3200318 ERROR heat.engine.resource yield from self.action_handler_task(action, args=handler_args) 2024-08-21 09:54:52.194 3200318 ERROR heat.engine.resource File "/usr/lib/python3.9/site-packages/heat/engine/resource.py", line 984, in action_handler_task 2024-08-21 09:54:52.194 3200318 ERROR heat.engine.resource done = check(handler_data) 2024-08-21 09:54:52.194 3200318 ERROR heat.engine.resource File "/usr/lib/python3.9/site-packages/heat/engine/resources/openstack/octavia/loadbalancer.py", line 169, in check_create_complete 2024-08-21 09:54:52.194 3200318 ERROR heat.engine.resource return self._check_status() 2024-08-21 09:54:52.194 3200318 ERROR heat.engine.resource File "/usr/lib/python3.9/site-packages/heat/engine/resources/openstack/octavia/octavia_base.py", line 29, in _check_status 2024-08-21 09:54:52.194 3200318 ERROR heat.engine.resource raise exception.ResourceInError(resource_status=status) 2024-08-21 09:54:52.194 3200318 ERROR heat.engine.resource heat.common.exception.ResourceInError: Went to status ERROR due to "Unknown" 2024-08-21 09:54:52.194 3200318 ERROR heat.engine.resource 2024-08-21 09:54:58.804 3200319 INFO heat.engine.scheduler [None req-610b24bd-7511-4d31-9f8c-b922162a7648 jouvin - - - - -] Task pause timed out 2024-08-21 09:54:59.340 3200318 INFO heat.engine.scheduler [None req-610b24bd-7511-4d31-9f8c-b922162a7648 jouvin - - - - -] Task pause timed out ----- that results in Magnum setting the state to CREATE_FAILED despite it is a transient state from Heat point of view and despite Heat stack creation continues successfully once the transient state is cleared and Magnum later updates the status to CREATE_COMPLETED. Unfortunately magnum-conductor log has nothing related to these events (it is filled with barbican_client related messages mainly). Any idea what could cause this behaviour? Best regards, Michel

2 3

[nova][cinder] Can not find device after change HBA
by Qi Xu (Fujitsu) 21 Aug '24

21 Aug '24

Hi everyone, I am currently using RHOSP 16.2.4 and launched a virtual machine (VM) using a volume. From the attach information, I confirmed that this volume was mounted on ports 1 and 2 through HBA1 and HBA2 on the compute node. When I checked the devices on the compute node, I found two devices, which is as expected. [root@overcloud-novacompute-0 heat-admin]# multipath -ll LUN-UUID dm-5 FUJITSU,ETERNUS_DXL size=10G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw `-+- policy='service-time 0' prio=10 status=active |- 3:0:2:1 sdl 8:176 active ready running `- 5:0:5:1 sdv 65:80 active ready running [root@overcloud-novacompute-0 heat-admin]# ll /dev/disk/by-path/ lrwxrwxrwx. 1 root root 9 Aug 20 08:51 fc-HBA1-port1-lun-0 -> ../../sdv lrwxrwxrwx. 1 root root 9 Aug 20 08:51 pci-0000:00:0a.0-fc-port1-lun-0 -> ../../sdv lrwxrwxrwx. 1 root root 9 Aug 20 08:51 fc-HBA2-port2-lun-0 -> ../../sdl lrwxrwxrwx. 1 root root 9 Aug 20 08:51 pci-0000:00:0b.0-fc-port2-lun-0 -> ../../sdl I then shut down the VM and the compute node, and replaced HBA1 with HBA3 on the compute node. Under these circumstances, the volume should be mounted on ports 1 and 2 through HBA3 and HBA2 respectively. In other words, I think I should be able to find the following devices. device1: fc-HBA3-port1-lun-0 -> ../../sdl pci-0000:00:0c.0-fc-port1-lun-0 -> ../../sdl device2: fc-HBA2-port2-lun-0 -> ../../sdv pci-0000:00:0b.0-fc-port2-lun-0 -> ../../sdv However, after restarting the compute node and the VM, although the connection and read/write operations are functioning normally, the compute node fails to find the volume on HBA3. [root@overcloud-novacompute-0 heat-admin]# multipath -ll LUN-UUID dm-5 FUJITSU,ETERNUS_DXL size=10G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw `-+- policy='service-time 0' prio=10 status=enabled `- 5:0:5:1 sdl 8:176 active ready running [root@overcloud-novacompute-0 heat-admin]# ll /dev/disk/by-path/ lrwxrwxrwx. 1 root root 9 Aug 20 08:51 fc-HBA2-port2-lun-0 -> ../../sdl lrwxrwxrwx. 1 root root 9 Aug 20 08:51 pci-0000:00:0b.0-fc-port2-lun-0 -> ../../sdl Could you please advise on how I can make the compute node find the volume on HBA3 in this scenario? And I would like to know how to change the Initiator from HBA1 to HBA3 in the initiator_target_map of DB without detach volume? Regards, Inori.

1 0

[Ironic] Unable to deploy bare metal due to partx partition failure
by Junya Noguchi (Fujitsu) 21 Aug '24

21 Aug '24

To whom it may concern A bare metal deployment with Ironic failed. Deploy Command $ openstack server create --flavor "flavor_name" --image "image_name" --nic net-id="net_id" --security-group "group_name" --key-name "key_name" "server_name" This command powers on a bare metal node that was previously in a powered-off state and boots the deployment image. An error occurs during the deployment process on the deployment image and the server status is error. $ openstack server list +--------------------------------------+-----------------+--------+----------+-------------------+---------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-----------------+--------+----------+-------------------+---------------------+ | "id" | "server_name " | ERROR | | "image_name" | "flavor_name" | +--------------------------------------+-----------------+--------+----------+-------------------+---------------------+ When I checked the journal log, I saw the following errors. Can you tell me how to resolve these errors to ensure a successful deployment? I look forward to your reply. 1. Partitions created by partx fail Aug 02 03:05:28 localhost.localdomain ironic-python-agent[1047]: 2024-08-02 03:05:28.468 1047 DEBUG oslo_concurrency.processutils [-] CMD "partx -av /dev/sda" returned: 1 in 0.004s execute /opt/ironic-python-agent/lib64/python3.9/site-packages/oslo_concurrency/processutils.py:428 Aug 02 03:05:28 localhost.localdomain ironic-python-agent[1047]: 2024-08-02 03:05:28.468 1047 DEBUG oslo_concurrency.processutils [-] 'partx -av /dev/sda' failed. Retrying. execute /opt/ironic-python-agent/lib64/python3.9/site-packages/oslo_concurrency/processutils.py:483 Aug 02 03:05:30 localhost.localdomain ironic-python-agent[1047]: 2024-08-02 03:05:30.190 1047 DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): partx -av /dev/sda execute /opt/ironic-python-agent/lib64/python3.9/site-packages/oslo_concurrency/processutils.py:390 Aug 02 03:05:30 localhost.localdomain ironic-python-agent[1047]: 2024-08-02 03:05:30.194 1047 DEBUG oslo_concurrency.processutils [-] CMD "partx -av /dev/sda" returned: 1 in 0.004s execute /opt/ironic-python-agent/lib64/python3.9/site-packages/oslo_concurrency/processutils.py:428 Aug 02 03:05:30 localhost.localdomain ironic-python-agent[1047]: 2024-08-02 03:05:30.195 1047 DEBUG oslo_concurrency.processutils [-] 'partx -av /dev/sda' failed. Retrying. execute /opt/ironic-python-agent/lib64/python3.9/site-packages/oslo_concurrency/processutils.py:483 Aug 02 03:05:30 localhost.localdomain systemd[1]: systemd-hostnamed.service: Deactivated successfully. Aug 02 03:05:31 localhost.localdomain ironic-python-agent[1047]: 2024-08-02 03:05:31.586 1047 DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): partx -av /dev/sda execute /opt/ironic-python-agent/lib64/python3.9/site-packages/oslo_concurrency/processutils.py:390 Aug 02 03:05:31 localhost.localdomain ironic-python-agent[1047]: 2024-08-02 03:05:31.591 1047 DEBUG oslo_concurrency.processutils [-] CMD "partx -av /dev/sda" returned: 1 in 0.004s execute /opt/ironic-python-agent/lib64/python3.9/site-packages/oslo_concurrency/processutils.py:428 Aug 02 03:05:31 localhost.localdomain ironic-python-agent[1047]: 2024-08-02 03:05:31.591 1047 DEBUG oslo_concurrency.processutils [-] 'partx -av /dev/sda' failed. Not Retrying. execute /opt/ironic-python-agent/lib64/python3.9/site-packages/oslo_concurrency/processutils.py:479 Aug 02 03:05:31 localhost.localdomain ironic-python-agent[1047]: 2024-08-02 03:05:31.591 1047 DEBUG ironic_lib.utils [-] Command stdout is: "partition: none, disk: /dev/sda, lower: 0, upper: 0 Aug 02 03:05:31 localhost.localdomain ironic-python-agent[1047]: /dev/sda: partition table type 'gpt' detected Aug 02 03:05:31 localhost.localdomain ironic-python-agent[1047]: range recount: max partno=2, lower=0, upper=0 Aug 02 03:05:31 localhost.localdomain ironic-python-agent[1047]: " _log /opt/ironic-python-agent/lib64/python3.9/site-packages/ironic_lib/utils.py:99 Aug 02 03:05:31 localhost.localdomain ironic-python-agent[1047]: 2024-08-02 03:05:31.591 1047 DEBUG ironic_lib.utils [-] Command stderr is: "partx: /dev/sda: adding partition #1 failed: Device or resource busy Aug 02 03:05:31 localhost.localdomain ironic-python-agent[1047]: partx: /dev/sda: adding partition #2 failed: Device or resource busy Aug 02 03:05:31 localhost.localdomain ironic-python-agent[1047]: partx: /dev/sda: error adding partitions 1-2 2. The boot efi partition is empty Aug 02 03:05:32 localhost.localdomain ironic-python-agent[1047]: 2024-08-02 03:05:32.525 1047 DEBUG ironic_python_agent.efi_utils [-] Looking for all efi files on /tmp/tmp1gqks5kg/boot/efi _get_efi_bootloaders /opt/ironic-python-agent/lib64/python3.9/site-packages/ironic_python_agent/efi_utils.py:241 Aug 02 03:05:32 localhost.localdomain ironic-python-agent[1047]: 2024-08-02 03:05:32.525 1047 DEBUG ironic_python_agent.efi_utils [-] efi files found in /tmp/tmp1gqks5kg/boot/efi : [] _get_efi_bootloaders /opt/ironic-python-agent/lib64/python3.9/site-packages/ironic_python_agent/efi_utils.py:245 Aug 02 03:05:32 localhost.localdomain ironic-python-agent[1047]: 2024-08-02 03:05:32.525 1047 WARNING ironic_python_agent.efi_utils [-] Empty EFI partition detected. OS in which the above events occurred ・CentOS Stream 9 ・Ubuntu 24 ・OpenSUSE 15 What I've tried (none of which worked) ・Reduce flavor disk size ・Vary local_gb in bare metal node properties Properties driver: irmc boot_interface: irmc-pxe console_interface: ipmitool-socat deplo y_interface: direct inspect_interface: no-inspect management_interface: irmc network_interface: flat power_interface: irmc raid_interfa ce: no-raid rescue_interface: no-rescue storage_interface: noop v endor_interface: no-vendor memory_mb: 64000 loca l_gb: 800 image_type: partition image_disk_format: raw image_chec ksum: None Create bare metal image command (ubuntu example) $ disk-image-create ubuntu baremetal dhcp-all-interfaces -a amd64 grub2 -o ubuntu_x86 Best regards, Junya Noguchi.

1 0

Heat Caracal: log file full of oslo_messaging._drivers.amqpdriver Expecting reply to msg/Received RPC response
by Michel Jouvin 20 Aug '24

20 Aug '24

Hi, On our cloud where Caracal Magnum/Heat is deployed, heat_engine log file is filled with many occurences of the following pair of messages (IDs change for every pair) when there is a cluster creation (and probably deletion) : 2024-08-20 09:21:54.458 78230 INFO oslo_messaging._drivers.amqpdriver [None req-bd38bf98-d978-482a-8758-62b1af72fa43 jouvin - - - - -] Expecting reply to msg 699a920fddb940 8a81fd799125835996 in queue reply_2d4fccf5059340818bbfa72887989a6e 2024-08-20 09:21:54.468 78230 INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 699a920fddb9408a81fd799125835996 Why these INFO messages from oslo that are polluting the logs? we have debug=false in Heat configuration. It doesn't seem to happen in Antelope that we run on another cloud. Any way to disable these messages? Best regards, Michel

2 4

[devstack][glance] glance-manage Error
by Josephine Seifert 20 Aug '24

20 Aug '24

Hi everyone, within my devstack I can execute cinder-manage db sync or version and everything works fine: ``` stack@devstack:~/devstack$ cinder-manage db version INFO dbcounter [-] Registered counter for database cinder DEBUG oslo_db.sqlalchemy.engines [-] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=304079) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/engines.py:342}} DEBUG dbcounter [-] [304079] Writer thread running {{(pid=304079) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}} INFO alembic.runtime.migration [-] Context impl MySQLImpl. INFO alembic.runtime.migration [-] Will assume non-transactional DDL. a8189e5afd9a ``` But when I try to do the same with glance-manage I get an Error: ``` stack@devstack:~/devstack$ glance-manage db version CRITICAL glance [-] Unhandled error: sqlalchemy.exc.NoSuchModuleError: Can't load plugin: sqlalchemy.plugins:dbcounter ERROR glance Traceback (most recent call last): ERROR glance File "/usr/bin/glance-manage", line 10, in <module> .... ``` my current version of sqlalchemy is: stack@devstack:~/devstack$ pip show sqlalchemy Name: SQLAlchemy Version: 1.4.31 Does anyone have an idea, how I can fix this Error? greetings Luzi

2 2