[victoria][cinder][nova] Extending attached volume

Lee Yarwood lyarwood at redhat.com
Fri Feb 19 12:32:03 UTC 2021


On Fri, 19 Feb 2021 at 11:53, Ammad Syed <syedammad83 at gmail.com> wrote:
>
> I have enabled debug in cinder. Getting below error in nova.
>
> 2021-02-19 11:17:25.143 117236 DEBUG cinder.volume.targets.tgt [req-746294ae-3693-41f0-af6a-2c50cc5e7bba 2af528fdf3244e15b4f3f8fcfc0889c5 890eb2b7d1b8488aa88de7c34d08817a - - -] StdOut from tgt-admin --update:  _do_tgt_update /usr/lib/python3/dist-packages/cinder/volume/targets/tgt.py:113
> 2021-02-19 11:17:25.143 117236 DEBUG cinder.volume.targets.tgt [req-746294ae-3693-41f0-af6a-2c50cc5e7bba 2af528fdf3244e15b4f3f8fcfc0889c5 890eb2b7d1b8488aa88de7c34d08817a - - -] StdErr from tgt-admin --update:  _do_tgt_update /usr/lib/python3/dist-packages/cinder/volume/targets/tgt.py:114
> 2021-02-19 11:17:25.172 117236 DEBUG cinder.compute.nova [req-746294ae-3693-41f0-af6a-2c50cc5e7bba 2af528fdf3244e15b4f3f8fcfc0889c5 890eb2b7d1b8488aa88de7c34d08817a - - -] Creating Keystone token plugin using URL: http://controller.rapidcompute.com:5000/v3/ novaclient /usr/lib/python3/dist-packages/cinder/compute/nova.py:106
> 2021-02-19 11:17:25.406 117236 ERROR cinder.compute.nova [req-746294ae-3693-41f0-af6a-2c50cc5e7bba 2af528fdf3244e15b4f3f8fcfc0889c5 890eb2b7d1b8488aa88de7c34d08817a - - -] Failed to notify nova on events: [{'name': 'volume-extended', 'server_uuid': '0e9512fb-fb18-4572-8aec-b7508c9f6cc0', 'tag': 'af80330c-f910-46cb-9654-3290e047fbed'}].: keystoneauth1.exceptions.http.BadRequest: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. (HTTP 400) (Request-ID: req-af93c9fa-a055-43e3-9f0f-afcf04ba1081)

The 400 here is being returned from keystone right? The above text is
part of the ValidationError exception there [1]. You should be able to
see req-af93c9fa-a055-43e3-9f0f-afcf04ba1081 in the logs there. Can
you ensure your cinder.conf [nova] section is correct and contains
project_domain_name, project_name, user_domain_name etc?

[1] https://opendev.org/openstack/keystone/src/branch/master/keystone/exception.py#L98-L104

> On Fri, Feb 19, 2021 at 4:31 PM Luigi Toscano <ltoscano at redhat.com> wrote:
>>
>> On Friday, 19 February 2021 11:27:42 CET Ammad Syed wrote:
>> > Hi,
>> >
>> > I am trying to extend attached volume from openstack client. Exporting the
>> > volume API version 3.62, its giving me below error.
>> >
>> > root at controller ~(keystone_ubuntu20_new)# openstack volume list
>> > volume version 3.62 is not in supported versions: 1, 2, 3
>> > Invalid volume client version '3.62'. must be one of: 2, 3
>>
>> This is sadly expected, the openstack volume subcommand has a few limitations
>> right now.
>>
>>
>> >
>> > However the attached volume extension from cinder cli was successful. Its
>> > shows updated volume size.
>> >
>> > root at controller ~(keystone_ubuntu20_new)# OS_VOLUME_API_VERSION=3.62 cinder
>> > extend test 13
>> >
>> > However the volume size was not updated in vm. Checking the cinder volume
>> > logs, found below error.
>> >
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova
>> > [req-93a79710-96df-4e4f-9a0c-d4457d289122 2af528fdf3244e15b4f3f8fcfc0889c5
>> > 890eb2b7d1b8488aa88de7c34d08817a - - -] Failed to notify nova on events:
>> > [{'name': 'volume-extended', 'server_uuid':
>> > '0e9512fb-fb18-4572-8aec-b7508c9f6cc0', 'tag':
>> > 'af80330c-f910-46cb-9654-3290e047fbed'}].:
>> > keystoneauth1.exceptions.http.BadRequest: Expecting to find domain in
>> > project. The server could not comply with the request since it is either
>> > malformed or otherwise incorrect. The client is assumed to be in error.
>> > (HTTP 400) (Request-ID: req-bb86ee85-f6ba-4135-ac55-5917c8658e5c)
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova Traceback (most
>> > recent call last):
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova   File
>> > "/usr/lib/python3/dist-packages/cinder/compute/nova.py", line 149, in
>> > _send_events
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova     response =
>> > nova.server_external_events.create(events)
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova   File
>> > "/usr/lib/python3/dist-packages/novaclient/v2/server_external_events.py",
>> > line 38, in create
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova     return
>> > self._create('/os-server-external-events', body, 'events',
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova   File
>> > "/usr/lib/python3/dist-packages/novaclient/base.py", line 363, in _create
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova     resp, body =
>> > self.api.client.post(url, body=body)
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova   File
>> > "/usr/lib/python3/dist-packages/keystoneauth1/adapter.py", line 401, in post
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova     return
>> > self.request(url, 'POST', **kwargs)
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova   File
>> > "/usr/lib/python3/dist-packages/novaclient/client.py", line 69, in request
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova     resp, body =
>> > super(SessionClient, self).request(url,
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova   File
>> > "/usr/lib/python3/dist-packages/keystoneauth1/adapter.py", line 554, in
>> > request
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova     resp =
>> > super(LegacyJsonAdapter, self).request(*args, **kwargs)
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova   File
>> > "/usr/lib/python3/dist-packages/keystoneauth1/adapter.py", line 257, in
>> > request
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova     return
>> > self.session.request(url, method, **kwargs)
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova   File
>> > "/usr/lib/python3/dist-packages/keystoneauth1/session.py", line 780, in
>> > request
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova     auth_headers =
>> > self.get_auth_headers(auth)
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova   File
>> > "/usr/lib/python3/dist-packages/keystoneauth1/session.py", line 1191, in
>> > get_auth_headers
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova     return
>> > auth.get_headers(self, **kwargs)
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova   File
>> > "/usr/lib/python3/dist-packages/keystoneauth1/plugin.py", line 95, in
>> > get_headers
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova     token =
>> > self.get_token(session)
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova   File
>> > "/usr/lib/python3/dist-packages/keystoneauth1/identity/base.py", line 88,
>> > in get_token
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova     return
>> > self.get_access(session).auth_token
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova   File
>> > "/usr/lib/python3/dist-packages/keystoneauth1/identity/base.py", line 134,
>> > in get_access
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova     self.auth_ref =
>> > self.get_auth_ref(session)
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova   File
>> > "/usr/lib/python3/dist-packages/keystoneauth1/identity/generic/base.py",
>> > line 208, in get_auth_ref
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova     return
>> > self._plugin.get_auth_ref(session, **kwargs)
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova   File
>> > "/usr/lib/python3/dist-packages/keystoneauth1/identity/v3/base.py", line
>> > 187, in get_auth_ref
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova     resp =
>> > session.post(token_url, json=body, headers=headers,
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova   File
>> > "/usr/lib/python3/dist-packages/keystoneauth1/session.py", line 1139, in
>> > post
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova     return
>> > self.request(url, 'POST', **kwargs)
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova   File
>> > "/usr/lib/python3/dist-packages/keystoneauth1/session.py", line 976, in
>> > request
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova     raise
>> > exceptions.from_response(resp, method, url)
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova
>> > keystoneauth1.exceptions.http.BadRequest: Expecting to find domain in
>> > project. The server could not comply with the request since it is either
>> > malformed or otherwise incorrect. The client is assumed to be in error.
>> > (HTTP 400) (Request-ID: req-bb86ee85-f6ba-4135-ac55-5917c8658e5c)
>> > 2021-02-19 10:23:25.787 22742 ERROR cinder.compute.nova
>> > 2021-02-19 10:23:25.791 22742 INFO cinder.message.api
>> > [req-93a79710-96df-4e4f-9a0c-d4457d289122 2af528fdf3244e15b4f3f8fcfc0889c5
>> > 890eb2b7d1b8488aa88de7c34d08817a - - -] Creating message record for
>> > request_id = req-93a79710-96df-4e4f-9a0c-d4457d289122
>> > 2021-02-19 10:23:25.807 22742 INFO cinder.volume.manager
>> > [req-93a79710-96df-4e4f-9a0c-d4457d289122 2af528fdf3244e15b4f3f8fcfc0889c5
>> > 890eb2b7d1b8488aa88de7c34d08817a - - -] Extend volume completed
>> > successfully.
>> >
>> > Any advice?
>>
>> It fails in nova, it seems. Let's see if nova people answers (I've extended
>> the topic).
>>
>>
>> --
>> Luigi
>>
>>
>>
>
>
> --
> Regards,
>
>
> Syed Ammad Ali




More information about the openstack-discuss mailing list