August 2025 - openstack-discuss - lists.openstack.org

[openstack helm] [Glance] Glance showing OSError: unable to receive chunked part when uploading image
by daniel890723＠gmail.com 08 Nov '25

08 Nov '25

Hello i'm testing using Openstack-helm to install all component but after the installation complete i can't upload the OS image through both openstack CLI or Horizon it stuck at Queueing and when i checking Glance Api log it shows 2024-11-07 07:31:19.362 7 INFO glance.api.v2.image_data [None req-2bd28444-196c-42e9-94ef-18443b58310e b39b42e27f8646acad1303711f1dc8b8 e4ab1a6192b9486da698efa311086c53 - - default default] Unable to create trust: no such option collect_timing in group [keystone_authtoken] Use the existing user token. 2024-11-07 07:31:19.400 7 ERROR glance.api.v2.image_data [None req-2bd28444-196c-42e9-94ef-18443b58310e b39b42e27f8646acad1303711f1dc8b8 e4ab1a6192b9486da698efa311086c53 - - default default] Failed to upload image data due to internal error: OSError: unable to receive chunked part 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi [None req-2bd28444-196c-42e9-94ef-18443b58310e b39b42e27f8646acad1303711f1dc8b8 e4ab1a6192b9486da698efa311086c53 - - default default] Caught error: unable to receive chunked part: OSError: unable to receive chunked part 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi Traceback (most recent call last): 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/wsgi.py", line 1302, in __call__ 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi action_result = self.dispatch(self.controller, action, 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/wsgi.py", line 1345, in dispatch 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi return method(*args, **kwargs) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/utils.py", line 415, in wrapped 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi return func(self, req, *args, **kwargs) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/api/v2/image_data.py", line 299, in upload 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi with excutils.save_and_reraise_exception(): 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi self.force_reraise() 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi raise self.value 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/api/v2/image_data.py", line 162, in upload 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi image.set_data(data, size, backend=backend) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/notifier.py", line 492, in set_data 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi with excutils.save_and_reraise_exception(): 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi self.force_reraise() 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi raise self.value 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/notifier.py", line 443, in set_data 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi self.repo.set_data(data, size, backend=backend, 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/quota/__init__.py", line 322, in set_data 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi self.image.set_data(data, size=size, backend=backend, 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/location.py", line 596, in set_data 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi self._upload_to_store(data, verifier, backend, size) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/location.py", line 487, in _upload_to_store 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi multihash, loc_meta) = self.store_api.add_with_multihash( 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/multi_backend.py", line 397, in add_with_multihash 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi return store_add_to_backend_with_multihash( 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/multi_backend.py", line 479, in store_add_to_backend_with_multihash 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi (location, size, checksum, multihash, metadata) = store.add( 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/driver.py", line 294, in add_adapter 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi metadata_dict) = store_add_fun(*args, **kwargs) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/capabilities.py", line 176, in op_checker 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi return store_op_fun(store, *args, **kwargs) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/_drivers/filesystem.py", line 764, in add 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi raise errors.get(e.errno, e) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/_drivers/filesystem.py", line 746, in add 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi for buf in utils.chunkreadable(image_file, 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance_store/common/utils.py", line 73, in chunkiter 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi chunk = fp.read(chunk_size) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/utils.py", line 294, in read 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi result = self.data.read(i) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/utils.py", line 121, in readfn 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi result = fd.read(*args) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/format_inspector.py", line 956, in read 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi chunk = self._source.read(size) 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi File "/var/lib/openstack/lib/python3.10/site-packages/glance/common/wsgi.py", line 1045, in read 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi data = uwsgi.chunked_read() 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi OSError: unable to receive chunked part 2024-11-07 07:31:19.426 7 ERROR glance.common.wsgi Ceph Cluster Status: NAME DATADIRHOSTPATH MONCOUNT AGE PHASE MESSAGE HEALTH EXTERNAL FSID ceph /var/lib/rook 3 82m Ready Cluster created successfully HEALTH_OK 5cf8b9bb-13e1-4c31-b47d-9e7d92c3c05d Ceph osd status: ID CLASS WEIGHT REWEIGHT SIZE RAW USE DATA OMAP META AVAIL %USE VAR PGS STATUS TYPE NAME -1 0.35156 - 360 GiB 3.3 GiB 493 MiB 0 B 2.8 GiB 357 GiB 0.91 1.00 - root default -5 0.11719 - 120 GiB 1.1 GiB 164 MiB 0 B 964 MiB 119 GiB 0.92 1.01 - host instance-lu-1 2 hdd 0.11719 1.00000 120 GiB 1.1 GiB 164 MiB 0 B 964 MiB 119 GiB 0.92 1.01 195 up osd.2 -3 0.11719 - 120 GiB 1.1 GiB 164 MiB 0 B 951 MiB 119 GiB 0.91 1.00 - host instance-lu-2 0 hdd 0.11719 1.00000 120 GiB 1.1 GiB 164 MiB 0 B 951 MiB 119 GiB 0.91 1.00 195 up osd.0 -7 0.11719 - 120 GiB 1.1 GiB 164 MiB 0 B 951 MiB 119 GiB 0.91 1.00 - host instance-lu-3 1 hdd 0.11719 1.00000 120 GiB 1.1 GiB 164 MiB 0 B 951 MiB 119 GiB 0.91 1.00 197 up osd.1 TOTAL 360 GiB 3.3 GiB 493 MiB 0 B 2.8 GiB 357 GiB 0.91 MIN/MAX VAR: 1.00/1.01 STDDEV: 0.01 now i'm able to create Empty Volume using horizon(can't create volume using default cirros image ) and i can start an Instance without volume and mount the empty volume in(Ceph also can detect when i use that volume) is there something i need to set when starting glance? because i'm following the Openstack-helm installation document to install openstack with helm

5 8

[openstack][kolla-ansible]about group node config overwrite
by Nguyễn Hữu Khôi 30 Oct '25

30 Oct '25

Hello Koller. I want to ask if we can overwrite config for a group of nodes. For example, compute01 >> compute10 will have the same nova configuration compute11 >> compute20 will have the same nova configuration. It will be very nice if we support this way. Thank you. Regards Nguyen Huu Khoi

4 7

[oslo][messaging] Container fix of QManager breaks it for containers
by Dmitriy Rabotyagov 22 Oct '25

22 Oct '25

Hey folks, I wanted to raise attention to a relatively old issue, which has had a patch around since January [1]. Basically a regression [2] was introduced and backported to 2024.1 as part of an attempt to fix QManager execution in containers. However, it does not account for the possibility of os.getpgrp() returning 0 in some edge cases, which makes, for instance, impossible to sync database for Nova during upgrades, if QManager is used. Thus I'm asking to kindly review the [1] as it appears to be a blocker for some upgrades involving affected versions of oslo.messaging. I have not spotted this patch right away, so I proposed an alternative [3], but I'm happy to abandon it in favor of [1]. Thanks in advance for upcoming reviews :) [1] https://review.opendev.org/c/openstack/oslo.messaging/+/939540 [2] https://review.opendev.org/c/openstack/oslo.messaging/+/928034 [3] https://review.opendev.org/c/openstack/oslo.messaging/+/958528

2 3

[Neutron][operators] LinuxBridge to OVN Outreach
by Daniel Failing 29 Sep '25

29 Sep '25

Hi all, Following on the good session we had last time discussing migrating out of LinuxBridge between operators, it was proposed to meet again in June. To find a good time slot, please respond to the following survey: https://newdle.cern.ch/newdle/TtbH8Dbv For reference, the notes from last meeting: https://etherpad.opendev.org/p/neutron-lb-ovn-migration Best, Daniel Failing CERN Cloud Infrastructure

2 3

[nova] Nova API call duration increases after upgrading to Caracal
by Chang Xue 22 Sep '25

22 Sep '25

Hello, we recently upgraded from Bobcat to Caracal. After upgrading, an interesting increment of duration of nova api call is observed. It's involved to multiple API calls, e.g. os-query-sets, /v2.1/servers/detail, etc. And it seems to be accumulated, meaning whenever nova-api is restarted, the duration seems to be fairly fast, giving response within like 80, 90 ms. As it keeps running for like up to 10 days, the duration increases to over 500 ms. I noticed that Caracal is using UnifiedLimitsDriver instead of DbQuotaDriver. I tried to change nova.conf to use DbQuotaDriver but it seems not working well with reducing the duration. I understand that calling to Keystone and Placement (and likely other external APIs) could result to longer duration, but pretty confused why it shows up as accumulated. Wondering if anyone else in Openstack community noticed the similar issue, thanks!

6 14

[cinder] IBM SVf driver callhome
by Vivek Pandey 10 Sep '25

10 Sep '25

Hi Joe/ Brian, Since we didn’t get a chance to discuss the patch [IBM SVf driver] Adding support for callhome plugin, we can continue the conversation here. Patch : https://review.opendev.org/c/openstack/cinder/+/951829. We need callhome enabled as default for which we have introduced a parameter “callhome” in cinder.conf. This feature is to fecth the os_version, cinder-manage version and ispbhaenabled. Note: We are not fetching any customer’s sensitive data. Please let me know if you’ve had a chance to review NetApp’s ActiveIQ implementation. Netapp’s ActiveIQ code ref : https://github.com/openstack/cinder/blob/master/cinder/volume/drivers/netap… As we have discussed during the 9th July cinder call, Anoop from NetApp confirmed that they are enabling it by default and EMS details are being captured from the logs. Based on that, do you think we can align or extend our current IBM callhome implementation (from the patch above) to follow a similar approach? Thanks, Vivek

5 9

[oslo] status of eventlet removal for oslo.service?
by Chris Friesen 05 Sep '25

05 Sep '25

Hi everyone, Looking at the "epoxy" spec for removing Eventlet from oslo.service (https://specs.openstack.org/openstack/oslo-specs/specs/epoxy/remove-eventle…) the very top line is a link to search blueprints.launchpad.net. Following that link it seems like there is no blueprint for removing eventlet from oslo.service, so does the spec itself (with the proposed milestones) represent the current plan? Separately, is there somewhere that shows the progress on the threading backend for oslo.service? I see it exists, are there any limitations on the current implementation? Thanks, Chris

2 1

[keystone][ops] Upgrade to 2025.1 and "corner cases"
by Dmitriy Rabotyagov 03 Sep '25

03 Sep '25

Hey folks, I wanted to raise awareness of operators regarding Keystone upgrade to Epoxy (2025.1), as found that existing release notes are slightly more vague about corner case definition. During analysis, it appeared that a lot of users (and under "a lot" I mean 4 figures number) in my really old deployments (which were deployed during Newton), still in use passwords hashed with sha512_crypt which has been dropped in 2025.1 [1]. As this was referenced as a "corner case" in upgrade docs, it may not bring enough attention to this matter reading notes, while sha512_crypt removal is mentioned in "Other" section which I personally missed when analyzing upgrade requirements. So I would highly recommend to ensure that all your users are using supported hashing algorithms before the upgrade to Epoxy, and inform them to reset their passwords if they don't. You can list users who are using unsupported hashing algorithm using following MySQL query: SELECT distinct l_u.user_id FROM keystone.local_user AS l_u JOIN keystone.password AS hash ON l_u.id = hash.local_user_id WHERE (hash.password_hash LIKE '$6$%' OR hash.password_hash LIKE '$5$%') AND hash.expires_at i s null; Hope this helps somebody to save some time in the future. [1] https://opendev.org/openstack/keystone/commit/4a4137d5f77ced3f4f011846fb9c2…

7 13

[Nova&Neutron] pagination returns an http, not https href link
by Chang Xue 03 Sep '25

03 Sep '25

Hello, we just migrated using new openstack CLI tools, and when using pagination for both Nova and Neutron, we noticed that they are returning http href link although they are expecting https, causing failure: ``` openstack port list Failure: Unable to establish connection to http://openstack....example.com:9696/v2.0/ports: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')). Retrying in 0.5s.1 retries left Unable to establish connection to http://openstack.....example.com:9696/v2.0/ports: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) ``` Is this a bug in Nova/Neutron? I tried other services like Heat, Keystone, etc, and they either returning https href links or just take relative path without schema/host. Thanks, Chang

3 10

[neutron][ovn] Logical Port Statistics
by James Denton 03 Sep '25

03 Sep '25

Hello all, Is anyone aware of a method to gather statistics (bytes in/out, packets, etc.) for logical router ports or similar when using ML2/OVN? I understand neutron-metering-agent parity is currently a feature gap, but I’m curious if there’s some OVN-native functionality that can provide those details. The ask is to provide a bytes-in/out counter for tenant routers, specifically on the external interface(s). Thanks, James Get Outlook for Mac <https://aka.ms/GetOutlookForMac>

3 3