[nova] random unauthorized errors

Ammad Syed syedammad83 at gmail.com
Mon Dec 20 10:10:34 UTC 2021


Hi,

I am using nova 23.0.0 and I am randomly facing unauthorized error sometime
when nova uses neutron or cinder phython3 client during listing of server
and creation of server. When I list server again, it works fine. I have
found below logs in nova-api.

2021-12-20 09:10:24.920 2689471 DEBUG nova.api.openstack.wsgi
[req-91ee9c88-e8ab-4e9b-8e1a-ca8a807c7e45 2af528fdf3244e15b4f3f8fcfc0889c5
890eb2b7d1b8488aa88de7c34d08817a - default default] Calling method '<bound
method ServersController.detail of
<nova.api.openstack.compute.servers.ServersController object at
0x7f840b6c5070>>' _process_stack
/usr/lib/python3/dist-packages/nova/api/openstack/wsgi.py:513
2021-12-20 09:10:24.925 2689471 DEBUG nova.compute.api
[req-91ee9c88-e8ab-4e9b-8e1a-ca8a807c7e45 2af528fdf3244e15b4f3f8fcfc0889c5
890eb2b7d1b8488aa88de7c34d08817a - default default] Searching by:
{'project_id': '890eb2b7d1b8488aa88de7c34d08817a', 'deleted': False}
get_all /usr/lib/python3/dist-packages/nova/compute/api.py:2857
2021-12-20 09:10:24.930 2689471 DEBUG oslo_concurrency.lockutils
[req-91ee9c88-e8ab-4e9b-8e1a-ca8a807c7e45 2af528fdf3244e15b4f3f8fcfc0889c5
890eb2b7d1b8488aa88de7c34d08817a - default default] Lock
"00000000-0000-0000-0000-000000000000" acquired by
"nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections"
:: waited 0.000s inner
/usr/lib/python3/dist-packages/oslo_concurrency/lockutils.py:355
2021-12-20 09:10:24.931 2689471 DEBUG oslo_concurrency.lockutils
[req-91ee9c88-e8ab-4e9b-8e1a-ca8a807c7e45 2af528fdf3244e15b4f3f8fcfc0889c5
890eb2b7d1b8488aa88de7c34d08817a - default def2021-12-20 09:10:24.548
2689471 INFO nova.osapi_compute.wsgi.server
[req-c5da1a3c-3c99-46c2-84e9-b504ccc5d156 2af528fdf3244e15b4f3f8fcfc0889c5
890eb2b7d1b8488aa88de7c34d08817a - default
default] 172.16.30.56 "GET /v2.1/ HTTP/1.1" status: 200 len: 800 time:
0.0048089
2021-12-20 09:10:24.697 2689471 DEBUG nova.osapi_compute.wsgi.server
[req-572eca81-8d3b-4524-89e5-e4e1124a7ac5 - - - - -] (2689471) accepted
('172.16.30.56', 52016) server /usr/lib/pytho
n3/dist-packages/eventlet/wsgi.py:992
2021-12-20 09:10:24.920 2689471 DEBUG nova.api.openstack.wsgi
[req-91ee9c88-e8ab-4e9b-8e1a-ca8a807c7e45 2af528fdf3244e15b4f3f8fcfc0889c5
890eb2b7d1b8488aa88de7c34d08817a - default defaul
t] Calling method '<bound method ServersController.detail of
<nova.api.openstack.compute.servers.ServersController object at
0x7f840b6c5070>>' _process_stack /usr/lib/python3/dist-packag
es/nova/api/openstack/wsgi.py:513
2021-12-20 09:10:24.925 2689471 DEBUG nova.compute.api
[req-91ee9c88-e8ab-4e9b-8e1a-ca8a807c7e45 2af528fdf3244e15b4f3f8fcfc0889c5
890eb2b7d1b8488aa88de7c34d08817a - default default] Sear
ching by: {'project_id': '890eb2b7d1b8488aa88de7c34d08817a', 'deleted':
False} get_all /usr/lib/python3/dist-packages/nova/compute/api.py:2857
2021-12-20 09:10:24.930 2689471 DEBUG oslo_concurrency.lockutils
[req-91ee9c88-e8ab-4e9b-8e1a-ca8a807c7e45 2af528fdf3244e15b4f3f8fcfc0889c5
890eb2b7d1b8488aa88de7c34d08817a - default def
ault] Lock "00000000-0000-0000-0000-000000000000" acquired by
"nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections"
:: waited 0.000s inner /usr/lib/python3/d
ist-packages/oslo_concurrency/lockutils.py:355
2021-12-20 09:10:24.931 2689471 DEBUG oslo_concurrency.lockutils
[req-91ee9c88-e8ab-4e9b-8e1a-ca8a807c7e45 2af528fdf3244e15b4f3f8fcfc0889c5
890eb2b7d1b8488aa88de7c34d08817a - default def
ault] Lock "00000000-0000-0000-0000-000000000000" released by
"nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections"
:: held 0.001s inner
/usr/lib/python3/dist-packages/oslo_concurrency/lockutils.py:367
2021-12-20 09:10:24.933 2689471 DEBUG oslo_concurrency.lockutils
[req-91ee9c88-e8ab-4e9b-8e1a-ca8a807c7e45 2af528fdf3244e15b4f3f8fcfc0889c5
890eb2b7d1b8488aa88de7c34d08817a - default default] Lock
"fc805771-7891-4095-8162-491cf824b02f" acquired by
"nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections"
:: waited 0.000s inner
/usr/lib/python3/dist-packages/oslo_concurrency/lockutils.py:355
2021-12-20 09:10:24.934 2689471 DEBUG oslo_concurrency.lockutils
[req-91ee9c88-e8ab-4e9b-8e1a-ca8a807c7e45 2af528fdf3244e15b4f3f8fcfc0889c5
890eb2b7d1b8488aa88de7c34d08817a - default default] Lock
"fc805771-7891-4095-8162-491cf824b02f" released by
"nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections"
:: held 0.001s inner
/usr/lib/python3/dist-packages/oslo_concurrency/lockutils.py:367
2021-12-20 09:10:25.019 2689471 DEBUG nova.compute.multi_cell_list
[req-91ee9c88-e8ab-4e9b-8e1a-ca8a807c7e45 2af528fdf3244e15b4f3f8fcfc0889c5
890eb2b7d1b8488aa88de7c34d08817a - default default] Listed batch of 13
results from cell out of 21 limit. Returned 13 total so far. do_query
/usr/lib/python3/dist-packages/nova/compute/multi_cell_list.py:371
2021-12-20 09:10:25.093 2689471 DEBUG oslo_concurrency.lockutils
[req-91ee9c88-e8ab-4e9b-8e1a-ca8a807c7e45 2af528fdf3244e15b4f3f8fcfc0889c5
890eb2b7d1b8488aa88de7c34d08817a - default default] Lock
"fc805771-7891-4095-8162-491cf824b02f" acquired by
"nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections"
:: waited 0.000s inner
/usr/lib/python3/dist-packages/oslo_concurrency/lockutils.py:355
2021-12-20 09:10:25.094 2689471 DEBUG oslo_concurrency.lockutils
[req-91ee9c88-e8ab-4e9b-8e1a-ca8a807c7e45 2af528fdf3244e15b4f3f8fcfc0889c5
890eb2b7d1b8488aa88de7c34d08817a - default default] Lock
"fc805771-7891-4095-8162-491cf824b02f" released by
"nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections"
:: held 0.000s inner
/usr/lib/python3/dist-packages/oslo_concurrency/lockutils.py:367
*2021-12-20 09:10:25.393 2689471 DEBUG neutronclient.v2_0.client
[req-91ee9c88-e8ab-4e9b-8e1a-ca8a807c7e45 2af528fdf3244e15b4f3f8fcfc0889c5
890eb2b7d1b8488aa88de7c34d08817a - default default] Error message:
{"error": {"code": 401, "title": "Unauthorized", "message": "The request
you have made requires authentication."}} _handle_fault_response
/usr/lib/python3/dist-packages/neutronclient/v2_0/client.py:258*
2021-12-20 09:10:25.394 2689471 ERROR nova.network.neutron
[req-91ee9c88-e8ab-4e9b-8e1a-ca8a807c7e45 2af528fdf3244e15b4f3f8fcfc0889c5
890eb2b7d1b8488aa88de7c34d08817a - default default] Neutron client was not
able to generate a valid admin token, please verify Neutron admin
credential located in nova.conf:
neutronclient.common.exceptions.Unauthorized: 401-{'error': {'code': 401,
'title': 'Unauthorized', 'message': 'The request you have made requires
authentication.'}}
2021-12-20 09:10:25.395 2689471 INFO nova.api.openstack.wsgi
[req-91ee9c88-e8ab-4e9b-8e1a-ca8a807c7e45 2af528fdf3244e15b4f3f8fcfc0889c5
890eb2b7d1b8488aa88de7c34d08817a - default default] HTTP exception thrown:
Networking client is experiencing an unauthorized exception.
2021-12-20 09:10:25.396 2689471 DEBUG nova.api.openstack.wsgi
[req-91ee9c88-e8ab-4e9b-8e1a-ca8a807c7e45 2af528fdf3244e15b4f3f8fcfc0889c5
890eb2b7d1b8488aa88de7c34d08817a - default default] Returning 400 to user:
Networking client is experiencing an unauthorized exception. __call__
/usr/lib/python3/dist-packages/nova/api/openstack/wsgi.py:927
2021-12-20 09:10:25.398 2689471 INFO nova.osapi_compute.wsgi.server
[req-91ee9c88-e8ab-4e9b-8e1a-ca8a807c7e45 2af528fdf3244e15b4f3f8fcfc0889c5
890eb2b7d1b8488aa88de7c34d08817a - default default] 172.16.30.56 "GET
/v2.1/890eb2b7d1b8488aa88de7c34d08817a/servers/detail?limit=21&project_id=890eb2b7d1b8488aa88de7c34d08817a&sort_dir=desc&sort_dir=desc&sort_dir=desc&sort_key=created_at&sort_key=display_name&sort_key=uuid
HTTP/1.1" status: 400 len: 529 time: 0.7002470

Is there anything that I need to change ?

-- 
Regards,


Syed Ammad Ali
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20211220/a587b983/attachment.htm>


More information about the openstack-discuss mailing list