Open Stack

 Trying to deploy a new instance in Self-Service network while I have 
 already deployed two others (one in provider and one in Self-Service) in 
 Ocata I see in dashboard the error "Cannot retrieve instances list"


 At the same time from CLI I am getting this:

 # nova list
 /usr/lib/python2.7/site-packages/novaclient/client.py:278: UserWarning: 
 The 'tenant_id' argument is deprecated in Ocata and its use may result 
 in errors in future releases. As 'project_id' is provided, the 
 'tenant_id' argument will be ignored.
   warnings.warn(msg)
 ERROR (ClientException): Unexpected API Error. Please report this at 
 http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
 <class 'sqlalchemy.exc.OperationalError'> (HTTP 500) (Request-ID: 
 req-f9da368f-0950-4e41-8973-a3fb8e82a133)


 or this

 # openstack server list
 Unexpected API Error. Please report this at 
 http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
 <class 'sqlalchemy.exc.OperationalError'> (HTTP 500) (Request-ID: 
 req-b426507f-9276-4fed-a403-36086e45b379)


 at the same time there are tons of these messages in nova-api.log:

 2017-03-14 00:29:26.355 3529 INFO nova.osapi_compute.wsgi.server 
 [req-2acb26b3-f975-4bb2-a3bf-599af7fb7930 - - - - -] 10.10.10.10 "GET 
 /v2.1/23078e37252f4437954ed19faabfa7e1/os-quota-sets/23078e37252f4437954ed19faabfa7e1 
 HTTP/1.1" status: 200 len: 704 time: 0.0377400
 2017-03-14 00:29:26.742 3529 INFO nova.osapi_compute.wsgi.server 
 [req-02d9e4c2-c085-4b13-ba08-e153a6ec1c65 - - - - -] 10.10.10.10 "GET 
 /v2.1/23078e37252f4437954ed19faabfa7e1/servers/detail?project_id=23078e37252f4437954ed19faabfa7e1&tenant_id=23078e37252f4437954ed19faabfa7e1 
 HTTP/1.1" status: 200 len: 347 time: 0.18465902017-03-14 00:29:26.355 
 3529 INFO nova.osapi_compute.wsgi.server 
 [req-2acb26b3-f975-4bb2-a3bf-599af7fb7930 - - - - -] 10.10.10.10 "GET 
 /v2.1/23078e37252f4437954ed19faabfa7e1/os-quota-sets/23078e37252f4437954ed19faabfa7e1 
 HTTP/1.1" status: 200 len: 704 time: 0.0377400
 2017-03-14 00:29:26.742 3529 INFO nova.osapi_compute.wsgi.server 
 [req-02d9e4c2-c085-4b13-ba08-e153a6ec1c65 - - - - -] 10.10.10.10 "GET 
 /v2.1/23078e37252f4437954ed19faabfa7e1/servers/detail?project_id=23078e37252f4437954ed19faabfa7e1&tenant_id=23078e37252f4437954ed19faabfa7e1 
 HTTP/1.1" status: 200 len: 347 time: 0.1846590
 2017-03-14 00:29:26.787 3529 INFO nova.osapi_compute.wsgi.server 
 [req-7e76fa0c-2f29-48e3-9aa1-13e0b655284f - - - - -] 10.10.10.10 "GET 
 /v2.1/23078e37252f4437954ed19faabfa7e1/flavors/detail HTTP/1.1" status: 
 200 len: 1305 time: 0.0396252
 2017-03-14 00:29:35.388 3529 INFO nova.osapi_compute.wsgi.server 
 [req-95f6061c-ef96-423e-9db9-a68597e18b37 - - - - -] 10.10.10.10 "GET 
 /v2.1/23078e37252f4437954ed19faabfa7e1/os-quota-sets/23078e37252f4437954ed19faabfa7e1 
 HTTP/1.1" status: 200 len: 704 time: 0.0361681
 2017-03-14 00:29:35.608 3529 INFO nova.osapi_compute.wsgi.server 
 [req-082511ed-303d-4e52-b877-e685d3b14651 - - - - -] 10.10.10.10 "GET 
 /v2.1/23078e37252f4437954ed19faabfa7e1/servers/detail?project_id=23078e37252f4437954ed19faabfa7e1&tenant_id=23078e37252f4437954ed19faabfa7e1 
 HTTP/1.1" status: 200 len: 347 time: 0.1575720
 2017-03-14 00:29:35.656 3529 INFO nova.osapi_compute.wsgi.server 
 [req-0f5333b2-af0c-4a86-8ce1-3a55cd7dbce6 - - - - -] 10.10.10.10 "GET 
 /v2.1/23078e37252f4437954ed19faabfa7e1/flavors/detail HTTP/1.1" status: 
 200 len: 1305 time: 0.0422440
 2017-03-14 00:29:40.204 3529 INFO nova.osapi_compute.wsgi.server 
 [req-5e834c54-0c34-44d1-a762-11aac291be5d - - - - -] 10.10.10.10 "GET 
 /v2.1/23078e37252f4437954ed19faabfa7e1/servers/detail?project_id=23078e37252f4437954ed19faabfa7e1 
 HTTP/1.1" status: 200 len: 347 time: 0.2034490
 2017-03-14 00:29:48.050 3529 INFO nova.osapi_compute.wsgi.server 
 [req-fa75fe0f-68d7-4434-bad1-e7a03426f902 - - - - -] 10.10.10.10 "GET 
 /v2.1/23078e37252f4437954ed19faabfa7e1 HTTP/1.1" status: 404 len: 331 
 time: 0.0134480
 2017-03-14 00:29:48.069 3529 INFO nova.osapi_compute.wsgi.server 
 [req-a753ddd4-47aa-44eb-aa8b-51b997abb5e5 - - - - -] 10.10.10.10 "GET 
 /v2.1/ HTTP/1.1" status: 200 len: 725 time: 0.0143502
 2017-03-14 00:29:48.137 3517 INFO nova.osapi_compute.wsgi.server 
 [req-d2062f00-ca89-4a88-a426-91a5020aa168 - - - - -] 10.10.10.10 "GET 
 /v2.1/23078e37252f4437954ed19faabfa7e1/os-simple-tenant-usage/23078e37252f4437954ed19faabfa7e1?start=2017-03-12T00:00:00&end=2017-03-13T23:59:59 
 HTTP/1.1" status: 200 len: 2991 time: 0.0628219
 2017-03-14 00:29:48.230 3517 INFO nova.osapi_compute.wsgi.server 
 [req-d21bf6b5-8cc1-4670-8eb4-cf8fa0fec0b4 - - - - -] 10.10.10.10 "GET 
 /v2.1/23078e37252f4437954ed19faabfa7e1/os-simple-tenant-usage/23078e37252f4437954ed19faabfa7e1?start=2017-03-12T00:00:00&end=2017-03-13T23:59:59&marker=f05cc3ec-18c3-455b-8188-306a55779f27 
 HTTP/1.1" status: 200 len: 354 time: 0.08722402017-03-14 00:29:26.787 
 3529 INFO nova.osapi_compute.wsgi.server 
 [req-7e76fa0c-2f29-48e3-9aa1-13e0b655284f - - - - -] 10.10.10.10 "GET 
 /v2.1/23078e37252f4437954ed19faabfa7e1/flavors/detail HTTP/1.1" status: 
 200 len: 1305 time: 0.0396252
 2017-03-14 00:29:35.388 3529 INFO nova.osapi_compute.wsgi.server 
 [req-95f6061c-ef96-423e-9db9-a68597e18b37 - - - - -] 10.10.10.10 "GET 
 /v2.1/23078e37252f4437954ed19faabfa7e1/os-quota-sets/23078e37252f4437954ed19faabfa7e1 
 HTTP/1.1" status: 200 len: 704 time: 0.0361681
 2017-03-14 00:29:35.608 3529 INFO nova.osapi_compute.wsgi.server 
 [req-082511ed-303d-4e52-b877-e685d3b14651 - - - - -] 10.10.10.10 "GET 
 /v2.1/23078e37252f4437954ed19faabfa7e1/servers/detail?project_id=23078e37252f4437954ed19faabfa7e1&tenant_id=23078e37252f4437954ed19faabfa7e1 
 HTTP/1.1" status: 200 len: 347 time: 0.1575720
 2017-03-14 00:29:35.656 3529 INFO nova.osapi_compute.wsgi.server 
 [req-0f5333b2-af0c-4a86-8ce1-3a55cd7dbce6 - - - - -] 10.10.10.10 "GET 
 /v2.1/23078e37252f4437954ed19faabfa7e1/flavors/detail HTTP/1.1" status: 
 200 len: 1305 time: 0.0422440
 2017-03-14 00:29:40.204 3529 INFO nova.osapi_compute.wsgi.server 
 [req-5e834c54-0c34-44d1-a762-11aac291be5d - - - - -] 10.10.10.10 "GET 
 /v2.1/23078e37252f4437954ed19faabfa7e1/servers/detail?project_id=23078e37252f4437954ed19faabfa7e1 
 HTTP/1.1" status: 200 len: 347 time: 0.2034490


 The only solution I found was to reboot the nodes (both controller and 
 compute).

 After reboot all instances were shown in Compute as "Shutoff".

 Has anyone seen something like this before??

 Best regards,

 G.