Open Stack

Do you see any error message is nova-api.log or api.log?

From: Clement Buisson [mailto:clement.buisson at lookout.com]
Sent: Wednesday, October 09, 2013 5:33 PM
To: rvaknin at redhat.com
Cc: Gangur, Hrushikesh (R & D HP Cloud); openstack
Subject: Re: [Openstack] "nova list" returns nothing

Did anyone experience the same issue?

On Wed, Oct 2, 2013 at 11:52 PM, Rami Vaknin <rvaknin at redhat.com<mailto:rvaknin at redhat.com>> wrote:
On 10/03/2013 03:12 AM, Clement Buisson wrote:
All these variables are correct, I just double checked them.
This is really strange because it was working fine and stopped working all of the sudden!

Your request says that you're not using the admin project (tenant):
"X-Auth-Project-Id: main"
It should be:
"X-Auth-Project-Id: admin"




On Wed, Oct 2, 2013 at 3:31 PM, Gangur, Hrushikesh (R & D HP Cloud) <hrushikesh.gangur at hp.com<mailto:hrushikesh.gangur at hp.com>> wrote:
Ensure these environment variable is set correctly. My guess is that your environment variable must be pointing to a different project:

export OS_USERNAME=Admin
export OS_PASSWORD=secretword
export OS_TENANT_NAME=AdminProject
export OS_AUTH_URL=http://<keystoneIPaddress>:5000/v2.0/
export OS_AUTH_STRATEGY=keystone

From: Clement Buisson [mailto:clement.buisson at lookout.com<mailto:clement.buisson at lookout.com>]
Sent: Wednesday, October 02, 2013 3:06 PM
To: rvaknin at redhat.com<mailto:rvaknin at redhat.com>
Cc: openstack
Subject: Re: [Openstack] "nova list" returns nothing

Hi Rami,

I checked in Nova DB and I can see all my instances there (select * from instances). On my compute nodes, I ran virsh list --all (got list for kvm instances) and pgrep kvm (got list of pids) and I got valid information about my instances.

I am also running these commands from the Nova-API server.
I tried your curl idea and here is the result:

curl -i http://127.0.0.1:8774/v2/ebdee2cb697449c9a374c006a8c2701f/servers -X GET -H "X-Auth-Project-Id: main" -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: 9b7d9b07f5b14793b1d24c5ca42bedd2"

HTTP/1.1 200 OK
X-Compute-Request-Id: req-9847157c-9d54-42fc-b5d0-3d46be491ef6
Content-Type: application/json
Content-Length: 15
Date: Wed, 02 Oct 2013 22:01:13 GMT

{"servers": []}

So weird that this array is returning empty data...
nova-api must be confused!

Thoughts?

On Wed, Oct 2, 2013 at 12:37 PM, Rami Vaknin <rvaknin at redhat.com<mailto:rvaknin at redhat.com>> wrote:
On 10/02/2013 09:35 PM, Clement Buisson wrote:

Hello,

I am a Folsom user, I have been using the "nova list" command for a while until couple weeks ago.
That command use to work just fine and one day it just stopped working.
I can run the command but nothing will show up.

Here is the output of my command:

nova --debug list

REQ: curl -i http://127.0.0.1:8774/v2/ebdee2cb697449c9a374c006a8c2701f/servers/detail -X GET -H "X-Auth-Project-Id: main" -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: f03bbc13f258473ab53399e3541a8e70"

connect: (127.0.0.1, 8774)
send: u'GET /v2/ebdee2cb697449c9a374c006a8c2701f/servers/detail HTTP/1.1\r\nHost: 127.0.0.1:8774<http://127.0.0.1:8774>\r\nx-auth-project-id: main\r\nx-auth-token: f03bbc13f258473ab53399e3541a8e70\r\naccept-encoding: gzip, deflate\r\naccept: application/json\r\nuser-agent: python-novaclient\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: X-Compute-Request-Id: req-002a752b-2f47-4cda-8f91-b9a822ef754b
header: Content-Type: application/json
header: Content-Length: 15
header: Date: Thu, 26 Sep 2013 18:07:05 GMT
RESP:{'status': '200', 'content-length': '15', 'content-location': u'http://127.0.0.1:8774/v2/ebdee2cb697449c9a374c006a8c2701f/servers/detail'<http://127.0.0.1:8774/v2/ebdee2cb697449c9a374c006a8c2701f/servers/detail%27>, 'x-compute-request-id': 'req-002a752b-2f47-4cda-8f91-b9a822ef754b', 'date': 'Thu, 26 Sep 2013 18:07:05 GMT', 'content-type': 'application/json'} {"servers": []}

I also tried with the option "all-tenant":
nova --debug list --all-tenants

REQ: curl -i http://127.0.0.1:8774/v2/ebdee2cb697449c9a374c006a8c2701f/servers/detail?all_tenants=1 -X GET -H "X-Auth-Project-Id: main" -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: 9b7d9b07f5b14793b1d24c5ca42bedd2"
connect: (127.0.0.1, 8774)
send: u'GET /v2/ebdee2cb697449c9a374c006a8c2701f/servers/detail?all_tenants=1 HTTP/1.1\r\nHost: 127.0.0.1:8774<http://127.0.0.1:8774>\r\nx-auth-project-id: main\r\nx-auth-token: 9b7d9b07f5b14793b1d24c5ca42bedd2\r\naccept-encoding: gzip, deflate\r\naccept: application/json\r\nuser-agent: python-novaclient\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: X-Compute-Request-Id: req-50e187e8-74b0-4342-a400-b364a61748e1
header: Content-Type: application/json
header: Content-Length: 15
header: Date: Wed, 02 Oct 2013 18:32:44 GMT
RESP:{'status': '200', 'content-length': '15', 'content-location': u'http://127.0.0.1:8774/v2/ebdee2cb697449c9a374c006a8c2701f/servers/detail?all_tenants=1', 'x-compute-request-id': 'req-50e187e8-74b0-4342-a400-b364a61748e1', 'date': 'Wed, 02 Oct 2013 18:32:44 GMT', 'content-type': 'application/json'} {"servers": []}

Same thing with euca2ools:

euca-describe-instances --debug

2013-09-26 11:07:14,205 euca2ools [DEBUG]:Method: POST
2013-09-26 11:07:14,205 euca2ools [DEBUG]:Path: /services/Cloud/
2013-09-26 11:07:14,205 euca2ools [DEBUG]:Data:
2013-09-26 11:07:14,205 euca2ools [DEBUG]:Headers: {}
2013-09-26 11:07:14,205 euca2ools [DEBUG]:Host: 127.0.0.1:8773<http://127.0.0.1:8773>
2013-09-26 11:07:14,206 euca2ools [DEBUG]:establishing HTTP connection: kwargs={}
2013-09-26 11:07:14,206 euca2ools [DEBUG]:Token: None
2013-09-26 11:07:14,206 euca2ools [DEBUG]:using _calc_signature_2
2013-09-26 11:07:14,206 euca2ools [DEBUG]:query string: AWSAccessKeyId=fbae4dab6bc045469c1f9f843d5d99b0&Action=DescribeInstances&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2013-09-26T18%3A07%3A14Z&Version=2010-08-31
2013-09-26 11:07:14,206 euca2ools [DEBUG]:string_to_sign: POST
127.0.0.1:8773<http://127.0.0.1:8773>
/services/Cloud/
AWSAccessKeyId=fbae4dab6bc045469c1f9f843d5d99b0&Action=DescribeInstances&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2013-09-26T18%3A07%3A14Z&Version=2010-08-31
2013-09-26 11:07:14,206 euca2ools [DEBUG]:len(b64)=44
2013-09-26 11:07:14,206 euca2ools [DEBUG]:base64 encoded digest: aoAKyibiBwFSbMMiUiEivQCvF6o/Cf5E4yM82LN3LfI=
2013-09-26 11:07:14,206 euca2ools [DEBUG]:query_string: AWSAccessKeyId=fbae4dab6bc045469c1f9f843d5d99b0&Action=DescribeInstances&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2013-09-26T18%3A07%3A14Z&Version=2010-08-31 Signature: aoAKyibiBwFSbMMiUiEivQCvF6o/Cf5E4yM82LN3LfI=
send: 'POST /services/Cloud/ HTTP/1.1\r\nHost: 127.0.0.1:8773<http://127.0.0.1:8773>\r\nAccept-Encoding: identity\r\nContent-Length: 231\r\nContent-Type: application/x-www-form-urlencoded; charset=UTF-8\r\nUser-Agent: Boto/2.2.2 (linux2)\r\n\r\nAWSAccessKeyId=fbae4dab6bc045469c1f9f843d5d99b0&Action=DescribeInstances&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2013-09-26T18%3A07%3A14Z&Version=2010-08-31&Signature=aoAKyibiBwFSbMMiUiEivQCvF6o%2FCf5E4yM82LN3LfI%3D'
reply: 'HTTP/1.1 200 OK\r\n'
header: Content-Type: text/xml
header: Content-Length: 206
header: Date: Thu, 26 Sep 2013 18:07:14 GMT
2013-09-26 11:07:14,264 euca2ools [DEBUG]:<?xml version="1.0" ?><DescribeInstancesResponse xmlns="http://ec2.amazonaws.com/doc/2010-08-31/"><requestId>req-53dfca36-14fc-497d-a37d-220e6072e903</requestId><reservationSet/></DescribeInstancesResponse>

I am getting a 200 from Keystone, so it's not an auth problem...
Do you have any clues about that one?

Weird,

I would check whether the instances appear in nova database.

In addition, I would also check that the instances really exist in the compute nodes by: "virsh list -a" or/and "pgrep qemu" (will return a pid per running instance).

More two dump questions:
Are you sure that the machine you're running that command is the nova api machine (the api calls are to 127.0.0.1, local machine)?
Can you run the same curl command again but with .../servers instead of ../servers/details (run it first by "nova --debug list" to get a new token, then run it manually with a changed uri/url)?


Thanks a lot!
Clément


_______________________________________________

Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Post to     : openstack at lists.openstack.org<mailto:openstack at lists.openstack.org>

Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack



--



Thanks,



Rami Vaknin, QE @ Red Hat, TLV, IL.



--
Thanks,
Clément



--
Thanks,
Clément




--



Thanks,



Rami Vaknin, QE @ Red Hat, TLV, IL.



--
Thanks,
Clément
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20131010/bfb079c8/attachment.html>