- openstack-discuss - lists.openstack.org

openstack client integration to fetch and provide OIDC access tokens (v3oidcaccesstoken)?
by Christian Rohmann 25 Jan '23

25 Jan '23

Hey openstack-discuss, while there is support for OpenID Connect and its various flows in the openstack client (https://docs.openstack.org/python-openstackclient/latest/cli/man/openstack.…) I would like to have the user authenticate only with central IdP login via a web page and then receive an access token and not have each user's openstack cli be a full OIDC client handling credentials and authenticating against the IdP via the users password itself. The tricky bit here is having good tooling for users to authenticate via the existing SSO and then to get and refresh tokens which are then fed to the openstack CLI. I was wondering if anybody knows of some nice integrations / plugins / hooks to make it easy for users to deal with the authentication (usually via some web site) and then to inject the token (v3oidcaccesstoken) into openstack-cli? I found that Fedcloud.eu (https://www.fedcloud.eu/) does something like this (see https://fedcloudclient.fedcloud.eu/usage.html#authentication) via OIDC-Agent. But most platforms making use of OIDC seem to configure the openstack client with client_id and secret and have it authenticate directly with the IdP. Regards, Christian

3 5

podman manifest error | status 404 | openstack wallaby
by Swogat Pradhan 25 Jan '23

25 Jan '23

Hi, I am currently trying DCN for my openstack and when running the deployment command for the edge site i am getting the following error: FATAL | Pre-fetch all the containers | dcn01-hci-0 | item= 172.25.201.68:8787/tripleomaster/openstack-etcd:current-tripleo | error={"ansible_loop_var": "prefetch_image", "changed": false, "msg": "Failed to pull image 172.25.201.68:8787/tripleomaster/openstack-etcd:current-tripleo", "prefetch_image": " 172.25.201.68:8787/tripleomaster/openstack-etcd:current-tripleo"} Can someone please guide me on how to fix this issue? With regards, Swogat Pradhan

1 0

[Glance] No weekly meeting this week
by Pranali Deore 24 Jan '23

24 Jan '23

Hello, Thursday, 26th Jan is public holiday in India and almost half of the team will not be around, so cancelling glance weekly meeting for this week. Thanks & Regards, Pranali

1 0

[horizon][xstatic] xstatic-font-awesome and xstatic-jquery.tablesorter leaving OpenStack
by Radomir Dopieralski 24 Jan '23

24 Jan '23

Hi everyone, a few years ago we have a adopted under the OpenStack umbrella a number of xstatic packages used by Horizon and its plugins, because they seemed to be unmaintained at the time, and we needed to keep them secure. A few weeks ago one of those packages, xstatic-font-awesome, got a new release by its original authors – the good people at the MoinMoin Wiki project, where the XStatic mechanism originated. That surprised us a little, and raised some concerns, since the release wasn't done using the OpenStack process, so we reached out to clear this situation. After discussion, we have decided to remove two xstatic packages: xstatic-font-awesome and xstatic-jquery.tablesorter from the OpenStack repositories and let them be maintained by the MoinMoin Wiki contributors instead, together with many other xstatic packages they use, so that there is no more confusion about who is responsible for them. We will continue to use pinned versions of those packages in Horizon, but we will no longer keep them as part of the OpenStack project. We will start the retirement process on the repositories for those packages shortly. -- Radomir Dopieralski

1 0

[charms] Release of charmhub Openstack dedicated ussuri, victoria, wallaby for Ubuntu 20.04 (focal) LTS
by Alex Kavanagh 24 Jan '23

24 Jan '23

Hello As the charm-guide [1] explains, OpenStack, Ceph, OVN and supporting charms moved from a single stable charm (for each OpenStack component) that supported multiple OpenStack releases, to individual tracks that each targets a specific release [2]. The current Zed release also recently [6] became stable. Today, we have (re)released updated individual charms to the ussuri/stable, victoria/stable and wallaby/stable tracks. These are derived from the stable 21.10 charms (that supported queens -> xena). We have also released the Ceph octopus/stable track to support ussuri and victoria, and the existing Ceph pacific (which was already stable) that supports wallaby (and xena). The full list of charms for the Xena, Victoria and Wallaby OpenStack systems, includiong Ceph, OVN, mysql8, hacluster, rabbitmq-server, vault and others can be seen in the docs at [3]. The ussuri, victoria, wallaby, and xena tracks have been designed to be upgradable from the 21.10 charms if you are running the associated OpenStack version, but please do read the docs [4] about how to go about upgrading charms. The relevant track (e.g. victoria/stable) is also able to upgrade from the previous version. Please again consult the docs [5] for upgrading advice. This was a significant piece of work by all those involved, so many, many thanks to all who contributed and made this happen! Thanks! [1] https://docs.openstack.org/charm-guide/latest/project/charm-delivery.html [2] https://juju.is/docs/olm/deploy-a-charm-from-charmhub#heading--specify-a-ch… [3] https://docs.openstack.org/charm-guide/latest/project/charm-delivery.html#c… [4] https://docs.openstack.org/project-deploy-guide/charm-deployment-guide/late… [5] https://docs.openstack.org/project-deploy-guide/charm-deployment-guide/late… [6] https://lists.openstack.org/pipermail/openstack-discuss/2023-January/031872… -- Alex Kavanagh OpenStack Engineering - Canonical Ltd

1 0

[neutron] CI meeting 24.01.2023 cancelled
by Slawek Kaplonski 24 Jan '23

24 Jan '23

Hi, Due to overlapping internal meeting this week I will not be able to chair neutron ci meeting. There is nothing critical to discuss there so with Rodolfo we decided to cancel that meeting this week. See You all on the CI meeting next week. -- Slawek Kaplonski Principal Software Engineer Red Hat

1 0

Error Nova Compute during snapshot
by Kaiser Wassilij 24 Jan '23

24 Jan '23

Hi, I start creating snapshot. Wherever the instance is, I start journalctl -f . The following error occurs in libvirtd # Ansible managed DISTRIB_ID="OSA" DISTRIB_RELEASE="25.2.0" DISTRIB_CODENAME="Yoga" DISTRIB_DESCRIPTION="OpenStack-Ansible" internal error: cannot update AppArmor profile 'libvirt-c6aa0368-8ae5-4fe4-8ae5-93a92329aa74' Jan 23 16:04:42 bc2bl13 libvirtd[1321612]: Unable to restore security label on /var/lib/nova/instances/snapshots/tmpyhucdu8x/37698cb66b8a44599601c5166902552c.delta Jan 23 16:09:37 bc2bl13 libvirtd[1321612]: invalid argument: disk vda does not have an active block job Jan 23 16:09:46 bc2bl13 libvirtd[1321612]: internal error: Child process (LIBVIRT_LOG_OUTPUTS=3:stderr /usr/lib/libvirt/virt-aa-helper -r -u libvirt-c6aa0368-8ae5-4fe4-8ae5-93a92329aa74) unexpected exit status 1: 2023-01-23 16:09:46.640+0000: 3744159: info : libvirt version: 8.0.0, package: 1ubuntu7.1~cloud0 (Openstack Ubuntu Testing Bot <openstack-testing-bot(a)ubuntu.com> Wed, 25 May 2023 libvirt version: 8.0.0, package: 1ubuntu7.1~cloud0 (Openstack Ubuntu Testing Bot <openstack-testing-bot(a)ubuntu.com> Wed, 25 May 2022 14:51:12 +0000) 2023-01-23 16:09:46.640+0000: 3744159: info : hostname: bc2bl13 2023-01-23 16:09:46.640+0000: 3744159: error : virDomainDiskDefMirrorParse:8800 : unsupported configuration: unknown mirror job type '' virt-aa-helper: error: could not parse XML virt-aa-helper: error: could not get VM definition Jan 23 16:09:46 bc2bl13 libvirtd[1321612]: internal error: cannot update AppArmor profile 'libvirt-c6aa0368-8ae5-4fe4-8ae5-93a92329aa74' Jan 23 16:09:46 bc2bl13 libvirtd[1321612]: Unable to restore security label on /var/lib/nova/instances/snapshots/tmp_a9upl2m/d3d15a89bd6a4dcb8255cfe50d2adb68.delta Jan 24 09:17:56 bc2bl13 libvirtd[1321612]: invalid argument: disk vda does not have an active block job Jan 24 09:18:09 bc2bl13 libvirtd[1321612]: internal error: Child process (LIBVIRT_LOG_OUTPUTS=3:stderr /usr/lib/libvirt/virt-aa-helper -r -u libvirt-c6aa0368-8ae5-4fe4-8ae5-93a92329aa74) unexpected exit status 1: 2023-01-24 09:18:09.865+0000: 4081179: info : info : libvirt version: 8.0.0, package: 1ubuntu7.1~cloud0 (Openstack Ubuntu Testing Bot <openstack-testing-bot(a)ubuntu.com> Wed, 25 May 2023 libvirt version: 8.0.0, package: 1ubuntu7.1~cloud0 (Openstack Ubuntu Testing Bot <openstack-testing-bot(a)ubuntu.com> Wed, 25 May 2022 14:51:12 +0000) 2023-01-24 09:18:09.865+0000: 4081179: info : hostname: bc2bl13 2023-01-24 09:18:09.865+0000: 4081179: error : virDomainDiskDefMirrorParse:8800 : unsupported configuration: unknown mirror job type '' virt-aa-helper: error: could not parse XML virt-aa-helper: error: could not get VM definition Jan 24 09:18:09 bc2bl13 libvirtd[1321612]: internal error: cannot update AppArmor profile 'libvirt-c6aa0368-8ae5-4fe4-8ae5-93a92329aa74' Jan 24 09:18:09 bc2bl13 libvirtd[1321612]: Unable to restore security label on /var/lib/nova/instances/snapshots/tmpv4gdu8fe/b0b8a1ac3ea64e3ca74363175cc17bd4.delta has anyone had the same error. What have you done?

1 0

[ironic] moving on
by Ruby Loo 24 Jan '23

24 Jan '23

Hi ironic'ers, I'm moving on to a non-Ironic, non-OpenStack world. I'm glad to have been on this journey. The ironic community has been, and continues to be, great -- welcoming and helpful. Keep up the great work and the big visions [1]. I look forward to Ironic taking over the world ;) Thanks to everyone that I encountered; YOU are what made it so special and fun! --ruby [1] https://docs.openstack.org/ironic/latest/contributor/vision.html

3 2

Snapshots disappear during saving
by Karera Tony 24 Jan '23

24 Jan '23

Dear Team, I am using Openstack Wallaby deployed using kolla-ansible. I installed Glance with the ceph backend and all was well. However when I create snapshots, they disappear when they are saved. Any idea on how to resolve this? Regards Tony Karera

2 4

Instance Snapshots disappear while being saved
by Karera Tony 24 Jan '23

24 Jan '23

Dear Team, I have deployed Openstack Wallby using kolla-ansible. I faced no challenge during the Installation. However when I create an instance snapshot, It disappears while being saved and below are the glance logs that seem to show errors. The Glance_backend_storage is ceph. Please assist 24/Jan/2023 06:25:10] "GET /v2/images?limit=1000&sort_key=created_at&sort_dir=desc HTTP/1.1" 200 5995 0.058022 2023-01-24 06:25:12.381 50 INFO glance.api.v2.image_data [req-7eebf051-b8b0-4dff-9b7e-8d5085d77b5d f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] Unable to create trust: no such option collect_timing in group [keystone_authtoken] Use the existing user token. 2023-01-24 06:25:12.470 50 WARNING glance_store._drivers.rbd [req-7eebf051-b8b0-4dff-9b7e-8d5085d77b5d f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] Since image size is zero we will be doing resize-before-write which will be slower than normal 2023-01-24 06:25:13.010 51 INFO eventlet.wsgi.server [req-b91e54ba-f3fa-4fad-ab7a-1ef1ad2750fb f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:13] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008496 2023-01-24 06:25:15.657 52 INFO eventlet.wsgi.server [req-c9fa16c8-232d-45ff-b425-55504f332597 f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:15] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008760 2023-01-24 06:25:16.404 50 ERROR glance_store._drivers.rbd [req-7eebf051-b8b0-4dff-9b7e-8d5085d77b5d f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] Failed to store image 57e5c3ee-6576-4cf7-a72a-2038c86456bc Store Exception invalid literal for int() with base 16: b'': eventlet.wsgi.ChunkReadError: invalid literal for int() with base 16: b'' 2023-01-24 06:25:16.629 50 ERROR glance.api.v2.image_data [req-7eebf051-b8b0-4dff-9b7e-8d5085d77b5d f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] Failed to upload image data due to HTTP error: webob.exc.HTTPBadRequest: invalid literal for int() with base 16: b'' 2023-01-24 06:25:16.697 50 INFO eventlet.wsgi.server [req-7eebf051-b8b0-4dff-9b7e-8d5085d77b5d f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] Traceback (most recent call last): File "/var/lib/kolla/venv/lib/python3.8/site-packages/eventlet/wsgi.py", line 604, in handle_one_response write(b''.join(towrite)) File "/var/lib/kolla/venv/lib/python3.8/site-packages/eventlet/wsgi.py", line 538, in write wfile.flush() File "/usr/lib/python3.8/socket.py", line 687, in write return self._sock.send(b) File "/var/lib/kolla/venv/lib/python3.8/site-packages/eventlet/greenio/base.py", line 396, in send return self._send_loop(self.fd.send, data, flags) File "/var/lib/kolla/venv/lib/python3.8/site-packages/eventlet/greenio/base.py", line 383, in _send_loop return send_method(data, *args) BrokenPipeError: [Errno 32] Broken pipe 2023-01-24 06:25:16.697 50 INFO eventlet.wsgi.server [req-7eebf051-b8b0-4dff-9b7e-8d5085d77b5d f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.30,10.10.13.28 - - [24/Jan/2023 06:25:16] "PUT /v2/images/57e5c3ee-6576-4cf7-a72a-2038c86456bc/file HTTP/1.1" 400 0 4.384073 2023-01-24 06:25:18.281 49 INFO eventlet.wsgi.server [req-dcd70adf-5dd4-4a15-abe0-f1e074030346 f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:18] "GET /v2/images?limit=1000&sort_key=created_at&sort_dir=desc HTTP/1.1" 200 5995 0.067056 2023-01-24 06:25:20.935 52 INFO eventlet.wsgi.server [req-065021ef-2fa4-456f-bad8-22e4a06fec9a f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:20] "GET /v2/images?limit=1000&sort_key=created_at&sort_dir=desc HTTP/1.1" 200 5995 0.064581 2023-01-24 06:25:23.367 50 INFO eventlet.wsgi.server [-] 10.10.13.27 - - [24/Jan/2023 06:25:23] "GET / HTTP/1.1" 300 1517 0.003469 2023-01-24 06:25:23.583 51 INFO eventlet.wsgi.server [req-a0a51dc6-01c5-4dff-afe9-6c67a279e7fa f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:23] "GET /v2/images?limit=1000&sort_key=created_at&sort_dir=desc HTTP/1.1" 200 5995 0.072727 2023-01-24 06:25:26.211 49 INFO eventlet.wsgi.server [req-052bc425-90c3-4e4e-94e3-15aba795c96b f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:26] "GET /v2/images?limit=1000&sort_key=created_at&sort_dir=desc HTTP/1.1" 200 5995 0.055309 2023-01-24 06:25:28.039 49 INFO glance.api.v2.image_data [req-7eebf051-b8b0-4dff-9b7e-8d5085d77b5d f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] Unable to create trust: no such option collect_timing in group [keystone_authtoken] Use the existing user token. 2023-01-24 06:25:28.168 49 WARNING glance_store._drivers.rbd [req-7eebf051-b8b0-4dff-9b7e-8d5085d77b5d f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] Since image size is zero we will be doing resize-before-write which will be slower than normal 2023-01-24 06:25:28.887 51 INFO eventlet.wsgi.server [req-4664a441-2fa6-41fa-b985-b6290712e597 f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:28] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008411 2023-01-24 06:25:31.541 52 INFO eventlet.wsgi.server [req-31a7d4fc-a266-40a7-9b0d-6ba1e8c0cf47 f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:31] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008032 2023-01-24 06:25:34.168 52 INFO eventlet.wsgi.server [req-d8e4513e-9945-4f0c-9b2d-a1e191982f0b f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:34] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.009528 2023-01-24 06:25:36.798 52 INFO eventlet.wsgi.server [req-02c6e354-6137-4f31-9cf3-a2e9c0258938 f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:36] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008272 2023-01-24 06:25:39.435 50 INFO eventlet.wsgi.server [req-a30d0de8-9b34-414f-81ef-9a3457cf2758 f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:39] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008886 2023-01-24 06:25:42.059 52 INFO eventlet.wsgi.server [req-409a24f7-7d8e-4bb8-8cab-ef730cf35478 f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:42] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008929 2023-01-24 06:25:44.709 52 INFO eventlet.wsgi.server [req-ebee0818-8afe-488e-a9f2-52bcdfb65983 f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:44] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008267 2023-01-24 06:25:47.346 52 INFO eventlet.wsgi.server [req-71ad7a1a-acee-40f1-a3ed-70fe64428644 f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:47] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008255 2023-01-24 06:25:50.005 50 INFO eventlet.wsgi.server [req-a69f1a44-7689-494c-9e84-c434a1ea9838 f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:50] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008565 2023-01-24 06:25:52.631 50 INFO eventlet.wsgi.server [req-08c7e2db-c60e-4cfe-ab0e-78269924f60d f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:52] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008409 2023-01-24 06:25:53.521 48 INFO eventlet.wsgi.server [-] 10.10.13.27 - - [24/Jan/2023 06:25:53] "GET / HTTP/1.1" 300 1517 0.001898 2023-01-24 06:25:55.271 51 INFO eventlet.wsgi.server [req-63d5645d-c9dc-4ec4-86b6-48305b05ebe8 f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:55] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.005122 2023-01-24 06:25:57.911 50 INFO eventlet.wsgi.server [req-67b0f1e9-89a1-4e34-8f95-132f32f9c478 f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:57] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.009235 2023-01-24 06:26:00.554 48 INFO eventlet.wsgi.server [req-cc7033b2-698b-4bd5-96cf-589d864485ff f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:26:00] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008637 2023-01-24 06:26:03.181 50 INFO eventlet.wsgi.server [req-26a2b823-8a26-49b8-9d79-af6bf36c0e2a f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:26:03] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.007718 2023-01-24 06:26:05.835 50 INFO eventlet.wsgi.server [req-1be8a06e-6c3a-4966-9031-a1548b6b0f7a f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:26:05] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008469 2023-01-24 06:26:08.483 50 INFO eventlet.wsgi.server [req-b24990c6-319d-4837-8128-406f49d38530 f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:26:08] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008319 2023-01-24 06:26:11.134 50 INFO eventlet.wsgi.server [req-5393b80e-5069-47ee-a1f0-ef7174c1e09b f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:26:11] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008237 2023-01-24 06:26:13.765 50 INFO eventlet.wsgi.server [req-fcbc3f21-ddb5-4b3f-869c-b489ace9a453 f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:26:13] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008331 2023-01-24 06:26:16.405 52 INFO eventlet.wsgi.server [req-20d98cec-4e1e-4582-aaa2-68043927ab7f f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:26:16] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008366 2023-01-24 06:26:19.028 50 INFO eventlet.wsgi.server [req-2f1bc312-00b8-4e92-91ac-a3ba1f951742 f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:26:19] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008365 2023-01-24 06:26:20.032 49 ERROR glance_store._drivers.rbd [req-7eebf051-b8b0-4dff-9b7e-8d5085d77b5d f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] Failed to store image 57e5c3ee-6576-4cf7-a72a-2038c86456bc Store Exception unexpected end of file while parsing chunked data: OSError: unexpected end of file while parsing chunked data 2023-01-24 06:26:20.443 49 ERROR glance.api.v2.image_data [req-7eebf051-b8b0-4dff-9b7e-8d5085d77b5d f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] Failed to upload image data due to internal error: OSError: unexpected end of file while parsing chunked data 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi [req-7eebf051-b8b0-4dff-9b7e-8d5085d77b5d f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] Caught error: unexpected end of file while parsing chunked data: OSError: unexpected end of file while parsing chunked data 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi Traceback (most recent call last): 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance/common/wsgi.py", line 1353, in __call__ 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi action_result = self.dispatch(self.controller, action, 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance/common/wsgi.py", line 1397, in dispatch 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi return method(*args, **kwargs) 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance/common/utils.py", line 416, in wrapped 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi return func(self, req, *args, **kwargs) 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance/api/v2/image_data.py", line 300, in upload 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi self._restore(image_repo, image) 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi self.force_reraise() 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi raise self.value 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance/api/v2/image_data.py", line 165, in upload 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi image.set_data(data, size, backend=backend) 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance/domain/proxy.py", line 208, in set_data 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi self.base.set_data(data, size, backend=backend, set_active=set_active) 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance/notifier.py", line 501, in set_data 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi _send_notification(notify_error, 'image.upload', msg) 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi self.force_reraise() 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi raise self.value 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance/notifier.py", line 447, in set_data 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi self.repo.set_data(data, size, backend=backend, 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance/api/policy.py", line 273, in set_data 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi return self.image.set_data(*args, **kwargs) 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance/quota/__init__.py", line 322, in set_data 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi self.image.set_data(data, size=size, backend=backend, 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance/location.py", line 567, in set_data 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi self._upload_to_store(data, verifier, backend, size) 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance/location.py", line 458, in _upload_to_store 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi multihash, loc_meta) = self.store_api.add_with_multihash( 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance_store/multi_backend.py", line 398, in add_with_multihash 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi return store_add_to_backend_with_multihash( 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance_store/multi_backend.py", line 480, in store_add_to_backend_with_multihash 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi (location, size, checksum, multihash, metadata) = store.add( 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance_store/driver.py", line 279, in add_adapter 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi metadata_dict) = store_add_fun(*args, **kwargs) 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance_store/capabilities.py", line 176, in op_checker 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi return store_op_fun(store, *args, **kwargs) 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance_store/_drivers/rbd.py", line 629, in add 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi raise exc 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance_store/_drivers/rbd.py", line 574, in add 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi for chunk in chunks: 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance_store/common/utils.py", line 73, in chunkiter 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi chunk = fp.read(chunk_size) 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance/common/utils.py", line 294, in read 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi result = self.data.read(i) 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance/common/utils.py", line 121, in readfn 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi result = fd.read(*args) 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance/common/format_inspector.py", line 658, in read 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi chunk = self._source.read(size) 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/eventlet/wsgi.py", line 221, in read 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi return self._chunked_read(self.rfile, length) 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/eventlet/wsgi.py", line 192, in _chunked_read 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi raise IOError("unexpected end of file while parsing chunked data") 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi OSError: unexpected end of file while parsing chunked data 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi 2023-01-24 06:26:20.570 49 INFO eventlet.wsgi.server [req-7eebf051-b8b0-4dff-9b7e-8d5085d77b5d f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] Traceback (most recent call last): File "/var/lib/kolla/venv/lib/python3.8/site-packages/eventlet/wsgi.py", line 604, in handle_one_response write(b''.join(towrite)) File "/var/lib/kolla/venv/lib/python3.8/site-packages/eventlet/wsgi.py", line 538, in write wfile.flush() File "/usr/lib/python3.8/socket.py", line 687, in write return self._sock.send(b) File "/var/lib/kolla/venv/lib/python3.8/site-packages/eventlet/greenio/base.py", line 396, in send return self._send_loop(self.fd.send, data, flags) File "/var/lib/kolla/venv/lib/python3.8/site-packages/eventlet/greenio/base.py", line 383, in _send_loop return send_method(data, *args) BrokenPipeError: [Errno 32] Broken pipe 2023-01-24 06:26:20.571 49 INFO eventlet.wsgi.server [req-7eebf051-b8b0-4dff-9b7e-8d5085d77b5d f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.30,10.10.13.28 - - [24/Jan/2023 06:26:20] "PUT /v2/images/57e5c3ee-6576-4cf7-a72a-2038c86456bc/file HTTP/1.1" 500 0 52.610723 2023-01-24 06:26:21.649 51 INFO eventlet.wsgi.server [req-9eea9898-f4e7-4cb1-b02c-019b56fa00a9 f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:26:21] "GET /v2/images?limit=1000&sort_key=created_at&sort_dir=desc HTTP/1.1" 200 4774 0.068947 2023-01-24 06:26:23.635 50 INFO eventlet.wsgi.server [-] 10.10.13.27 - - [24/Jan/2023 06:26:23] "GET / HTTP/1.1" 300 1517 0.001586 ^C root@controller1:/home/stack#24/Jan/2023 06:25:10] "GET /v2/images?limit=1000&sort_key=created_at&sort_dir=desc HTTP/1.1" 200 5995 0.058022 2023-01-24 06:25:12.381 50 INFO glance.api.v2.image_data [req-7eebf051-b8b0-4dff-9b7e-8d5085d77b5d f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] Unable to create trust: no such option collect_timing in group [keystone_authtoken] Use the existing user token. 2023-01-24 06:25:12.470 50 WARNING glance_store._drivers.rbd [req-7eebf051-b8b0-4dff-9b7e-8d5085d77b5d f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] Since image size is zero we will be doing resize-before-write which will be slower than normal 2023-01-24 06:25:13.010 51 INFO eventlet.wsgi.server [req-b91e54ba-f3fa-4fad-ab7a-1ef1ad2750fb f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:13] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008496 2023-01-24 06:25:15.657 52 INFO eventlet.wsgi.server [req-c9fa16c8-232d-45ff-b425-55504f332597 f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:15] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008760 2023-01-24 06:25:16.404 50 ERROR glance_store._drivers.rbd [req-7eebf051-b8b0-4dff-9b7e-8d5085d77b5d f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] Failed to store image 57e5c3ee-6576-4cf7-a72a-2038c86456bc Store Exception invalid literal for int() with base 16: b'': eventlet.wsgi.ChunkReadError: invalid literal for int() with base 16: b'' 2023-01-24 06:25:16.629 50 ERROR glance.api.v2.image_data [req-7eebf051-b8b0-4dff-9b7e-8d5085d77b5d f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] Failed to upload image data due to HTTP error: webob.exc.HTTPBadRequest: invalid literal for int() with base 16: b'' 2023-01-24 06:25:16.697 50 INFO eventlet.wsgi.server [req-7eebf051-b8b0-4dff-9b7e-8d5085d77b5d f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] Traceback (most recent call last): File "/var/lib/kolla/venv/lib/python3.8/site-packages/eventlet/wsgi.py", line 604, in handle_one_response write(b''.join(towrite)) File "/var/lib/kolla/venv/lib/python3.8/site-packages/eventlet/wsgi.py", line 538, in write wfile.flush() File "/usr/lib/python3.8/socket.py", line 687, in write return self._sock.send(b) File "/var/lib/kolla/venv/lib/python3.8/site-packages/eventlet/greenio/base.py", line 396, in send return self._send_loop(self.fd.send, data, flags) File "/var/lib/kolla/venv/lib/python3.8/site-packages/eventlet/greenio/base.py", line 383, in _send_loop return send_method(data, *args) BrokenPipeError: [Errno 32] Broken pipe 2023-01-24 06:25:16.697 50 INFO eventlet.wsgi.server [req-7eebf051-b8b0-4dff-9b7e-8d5085d77b5d f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.30,10.10.13.28 - - [24/Jan/2023 06:25:16] "PUT /v2/images/57e5c3ee-6576-4cf7-a72a-2038c86456bc/file HTTP/1.1" 400 0 4.384073 2023-01-24 06:25:18.281 49 INFO eventlet.wsgi.server [req-dcd70adf-5dd4-4a15-abe0-f1e074030346 f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:18] "GET /v2/images?limit=1000&sort_key=created_at&sort_dir=desc HTTP/1.1" 200 5995 0.067056 2023-01-24 06:25:20.935 52 INFO eventlet.wsgi.server [req-065021ef-2fa4-456f-bad8-22e4a06fec9a f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:20] "GET /v2/images?limit=1000&sort_key=created_at&sort_dir=desc HTTP/1.1" 200 5995 0.064581 2023-01-24 06:25:23.367 50 INFO eventlet.wsgi.server [-] 10.10.13.27 - - [24/Jan/2023 06:25:23] "GET / HTTP/1.1" 300 1517 0.003469 2023-01-24 06:25:23.583 51 INFO eventlet.wsgi.server [req-a0a51dc6-01c5-4dff-afe9-6c67a279e7fa f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:23] "GET /v2/images?limit=1000&sort_key=created_at&sort_dir=desc HTTP/1.1" 200 5995 0.072727 2023-01-24 06:25:26.211 49 INFO eventlet.wsgi.server [req-052bc425-90c3-4e4e-94e3-15aba795c96b f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:26] "GET /v2/images?limit=1000&sort_key=created_at&sort_dir=desc HTTP/1.1" 200 5995 0.055309 2023-01-24 06:25:28.039 49 INFO glance.api.v2.image_data [req-7eebf051-b8b0-4dff-9b7e-8d5085d77b5d f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] Unable to create trust: no such option collect_timing in group [keystone_authtoken] Use the existing user token. 2023-01-24 06:25:28.168 49 WARNING glance_store._drivers.rbd [req-7eebf051-b8b0-4dff-9b7e-8d5085d77b5d f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] Since image size is zero we will be doing resize-before-write which will be slower than normal 2023-01-24 06:25:28.887 51 INFO eventlet.wsgi.server [req-4664a441-2fa6-41fa-b985-b6290712e597 f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:28] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008411 2023-01-24 06:25:31.541 52 INFO eventlet.wsgi.server [req-31a7d4fc-a266-40a7-9b0d-6ba1e8c0cf47 f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:31] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008032 2023-01-24 06:25:34.168 52 INFO eventlet.wsgi.server [req-d8e4513e-9945-4f0c-9b2d-a1e191982f0b f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:34] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.009528 2023-01-24 06:25:36.798 52 INFO eventlet.wsgi.server [req-02c6e354-6137-4f31-9cf3-a2e9c0258938 f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:36] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008272 2023-01-24 06:25:39.435 50 INFO eventlet.wsgi.server [req-a30d0de8-9b34-414f-81ef-9a3457cf2758 f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:39] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008886 2023-01-24 06:25:42.059 52 INFO eventlet.wsgi.server [req-409a24f7-7d8e-4bb8-8cab-ef730cf35478 f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:42] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008929 2023-01-24 06:25:44.709 52 INFO eventlet.wsgi.server [req-ebee0818-8afe-488e-a9f2-52bcdfb65983 f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:44] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008267 2023-01-24 06:25:47.346 52 INFO eventlet.wsgi.server [req-71ad7a1a-acee-40f1-a3ed-70fe64428644 f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:47] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008255 2023-01-24 06:25:50.005 50 INFO eventlet.wsgi.server [req-a69f1a44-7689-494c-9e84-c434a1ea9838 f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:50] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008565 2023-01-24 06:25:52.631 50 INFO eventlet.wsgi.server [req-08c7e2db-c60e-4cfe-ab0e-78269924f60d f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:52] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008409 2023-01-24 06:25:53.521 48 INFO eventlet.wsgi.server [-] 10.10.13.27 - - [24/Jan/2023 06:25:53] "GET / HTTP/1.1" 300 1517 0.001898 2023-01-24 06:25:55.271 51 INFO eventlet.wsgi.server [req-63d5645d-c9dc-4ec4-86b6-48305b05ebe8 f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:55] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.005122 2023-01-24 06:25:57.911 50 INFO eventlet.wsgi.server [req-67b0f1e9-89a1-4e34-8f95-132f32f9c478 f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:25:57] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.009235 2023-01-24 06:26:00.554 48 INFO eventlet.wsgi.server [req-cc7033b2-698b-4bd5-96cf-589d864485ff f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:26:00] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008637 2023-01-24 06:26:03.181 50 INFO eventlet.wsgi.server [req-26a2b823-8a26-49b8-9d79-af6bf36c0e2a f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:26:03] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.007718 2023-01-24 06:26:05.835 50 INFO eventlet.wsgi.server [req-1be8a06e-6c3a-4966-9031-a1548b6b0f7a f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:26:05] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008469 2023-01-24 06:26:08.483 50 INFO eventlet.wsgi.server [req-b24990c6-319d-4837-8128-406f49d38530 f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:26:08] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008319 2023-01-24 06:26:11.134 50 INFO eventlet.wsgi.server [req-5393b80e-5069-47ee-a1f0-ef7174c1e09b f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:26:11] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008237 2023-01-24 06:26:13.765 50 INFO eventlet.wsgi.server [req-fcbc3f21-ddb5-4b3f-869c-b489ace9a453 f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:26:13] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008331 2023-01-24 06:26:16.405 52 INFO eventlet.wsgi.server [req-20d98cec-4e1e-4582-aaa2-68043927ab7f f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:26:16] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008366 2023-01-24 06:26:19.028 50 INFO eventlet.wsgi.server [req-2f1bc312-00b8-4e92-91ac-a3ba1f951742 f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:26:19] "GET /v2/schemas/image HTTP/1.1" 200 6259 0.008365 2023-01-24 06:26:20.032 49 ERROR glance_store._drivers.rbd [req-7eebf051-b8b0-4dff-9b7e-8d5085d77b5d f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] Failed to store image 57e5c3ee-6576-4cf7-a72a-2038c86456bc Store Exception unexpected end of file while parsing chunked data: OSError: unexpected end of file while parsing chunked data 2023-01-24 06:26:20.443 49 ERROR glance.api.v2.image_data [req-7eebf051-b8b0-4dff-9b7e-8d5085d77b5d f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] Failed to upload image data due to internal error: OSError: unexpected end of file while parsing chunked data 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi [req-7eebf051-b8b0-4dff-9b7e-8d5085d77b5d f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] Caught error: unexpected end of file while parsing chunked data: OSError: unexpected end of file while parsing chunked data 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi Traceback (most recent call last): 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance/common/wsgi.py", line 1353, in __call__ 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi action_result = self.dispatch(self.controller, action, 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance/common/wsgi.py", line 1397, in dispatch 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi return method(*args, **kwargs) 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance/common/utils.py", line 416, in wrapped 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi return func(self, req, *args, **kwargs) 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance/api/v2/image_data.py", line 300, in upload 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi self._restore(image_repo, image) 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi self.force_reraise() 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi raise self.value 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance/api/v2/image_data.py", line 165, in upload 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi image.set_data(data, size, backend=backend) 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance/domain/proxy.py", line 208, in set_data 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi self.base.set_data(data, size, backend=backend, set_active=set_active) 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance/notifier.py", line 501, in set_data 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi _send_notification(notify_error, 'image.upload', msg) 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi self.force_reraise() 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi raise self.value 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance/notifier.py", line 447, in set_data 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi self.repo.set_data(data, size, backend=backend, 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance/api/policy.py", line 273, in set_data 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi return self.image.set_data(*args, **kwargs) 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance/quota/__init__.py", line 322, in set_data 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi self.image.set_data(data, size=size, backend=backend, 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance/location.py", line 567, in set_data 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi self._upload_to_store(data, verifier, backend, size) 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance/location.py", line 458, in _upload_to_store 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi multihash, loc_meta) = self.store_api.add_with_multihash( 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance_store/multi_backend.py", line 398, in add_with_multihash 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi return store_add_to_backend_with_multihash( 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance_store/multi_backend.py", line 480, in store_add_to_backend_with_multihash 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi (location, size, checksum, multihash, metadata) = store.add( 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance_store/driver.py", line 279, in add_adapter 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi metadata_dict) = store_add_fun(*args, **kwargs) 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance_store/capabilities.py", line 176, in op_checker 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi return store_op_fun(store, *args, **kwargs) 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance_store/_drivers/rbd.py", line 629, in add 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi raise exc 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance_store/_drivers/rbd.py", line 574, in add 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi for chunk in chunks: 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance_store/common/utils.py", line 73, in chunkiter 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi chunk = fp.read(chunk_size) 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance/common/utils.py", line 294, in read 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi result = self.data.read(i) 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance/common/utils.py", line 121, in readfn 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi result = fd.read(*args) 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/glance/common/format_inspector.py", line 658, in read 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi chunk = self._source.read(size) 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/eventlet/wsgi.py", line 221, in read 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi return self._chunked_read(self.rfile, length) 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.8/site-packages/eventlet/wsgi.py", line 192, in _chunked_read 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi raise IOError("unexpected end of file while parsing chunked data") 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi OSError: unexpected end of file while parsing chunked data 2023-01-24 06:26:20.563 49 ERROR glance.common.wsgi 2023-01-24 06:26:20.570 49 INFO eventlet.wsgi.server [req-7eebf051-b8b0-4dff-9b7e-8d5085d77b5d f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] Traceback (most recent call last): File "/var/lib/kolla/venv/lib/python3.8/site-packages/eventlet/wsgi.py", line 604, in handle_one_response write(b''.join(towrite)) File "/var/lib/kolla/venv/lib/python3.8/site-packages/eventlet/wsgi.py", line 538, in write wfile.flush() File "/usr/lib/python3.8/socket.py", line 687, in write return self._sock.send(b) File "/var/lib/kolla/venv/lib/python3.8/site-packages/eventlet/greenio/base.py", line 396, in send return self._send_loop(self.fd.send, data, flags) File "/var/lib/kolla/venv/lib/python3.8/site-packages/eventlet/greenio/base.py", line 383, in _send_loop return send_method(data, *args) BrokenPipeError: [Errno 32] Broken pipe 2023-01-24 06:26:20.571 49 INFO eventlet.wsgi.server [req-7eebf051-b8b0-4dff-9b7e-8d5085d77b5d f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.30,10.10.13.28 - - [24/Jan/2023 06:26:20] "PUT /v2/images/57e5c3ee-6576-4cf7-a72a-2038c86456bc/file HTTP/1.1" 500 0 52.610723 2023-01-24 06:26:21.649 51 INFO eventlet.wsgi.server [req-9eea9898-f4e7-4cb1-b02c-019b56fa00a9 f75f32a3c1fd4cf68cdb7b76d70ee9a8 bf5353fb17604156b492d5a7f00992ff - default default] 10.10.13.27,10.10.13.28 - - [24/Jan/2023 06:26:21] "GET /v2/images?limit=1000&sort_key=created_at&sort_dir=desc HTTP/1.1" 200 4774 0.068947 2023-01-24 06:26:23.635 50 INFO eventlet.wsgi.server [-] 10.10.13.27 - - [24/Jan/2023 06:26:23] "GET / HTTP/1.1" 300 1517 0.001586 ^C root@controller1:/home/stack# Regards Tony Karera

1 0