Open Stack

Tue Apr 30 18:13:06 UTC 2013

When i try to list the volumes this is what i see in the cinder api logs
file:

2013-04-30 17:43:07    DEBUG [keystoneclient.middleware.auth_token]
Authenticating user token
2013-04-30 17:43:07    DEBUG [keystoneclient.middleware.auth_token]
Removing headers from request environment:
X-Identity-Status,X-Domain-Id,X-Domain-Name,X-Project-Id,X-Project-Name,X-Project-Domain-Id,X-Project-Domain-Name,X-User-Id,X-User-Name,X-User-Domain-Id,X-User-Domain-Name,X-Roles,X-Service-Catalog,X-User,X-Tenant-Id,X-Tenant-Name,X-Tenant,X-Role
2013-04-30 17:43:07    ERROR [keystoneclient.common.cms] Verify error:
Verification failure

140606277047968:error:0407006A:rsa
routines:RSA_padding_check_PKCS1_type_1:block type is not 01:rsa_pk1.c:100:
140606277047968:error:04067072:rsa routines:RSA_EAY_PUBLIC_DECRYPT:padding
check failed:rsa_eay.c:721:
140606277047968:error:2E09A09E:CMS
routines:CMS_SignerInfo_verify_content:verification failure:cms_sd.c:900:
140606277047968:error:2E09D06D:CMS routines:CMS_verify:content verify
error:cms_smime.c:425:

2013-04-30 17:43:07    DEBUG [keystoneclient.middleware.auth_token] Token
validation failure.
Traceback (most recent call last):
  File
"/usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py",
line 688, in _validate_user_token
    verified = self.verify_signed_token(user_token)
  File
"/usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py",
line 1043, in verify_signed_token
    if self.is_signed_token_revoked(signed_text):
  File
"/usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py",
line 1007, in is_signed_token_revoked
    revocation_list = self.token_revocation_list
  File
"/usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py",
line 1079, in token_revocation_list
    self.token_revocation_list = self.fetch_revocation_list()
  File
"/usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py",
line 1109, in fetch_revocation_list
    return self.cms_verify(data['signed'])
  File
"/usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py",
line 1038, in cms_verify
    raise err
CalledProcessError: Command 'openssl' returned non-zero exit status 4
2013-04-30 17:43:07    DEBUG [keystoneclient.middleware.auth_token] Marking
token MIIMbwYJKoZIhvcNA ... Od7Wrw6Aw== as unauthorized in memcache
2013-04-30 17:43:07  WARNING [keystoneclient.middleware.auth_token]
Authorization failed for token MIIMbwYJKoZIhvcNA ... Od7Wrw6Aw==
2013-04-30 17:43:07     INFO [keystoneclient.middleware.auth_token] Invalid
user token - rejecting request
2013-04-30 17:43:07    DEBUG [keystoneclient.middleware.auth_token]
Authenticating user token
2013-04-30 17:43:07    DEBUG [keystoneclient.middleware.auth_token]
Removing headers from request environment:
X-Identity-Status,X-Domain-Id,X-Domain-Name,X-Project-Id,X-Project-Name,X-Project-Domain-Id,X-Project-Domain-Name,X-User-Id,X-User-Name,X-User-Domain-Id,X-User-Domain-Name,X-Roles,X-Service-Catalog,X-User,X-Tenant-Id,X-Tenant-Name,X-Tenant,X-Role
2013-04-30 17:43:07    ERROR [keystoneclient.common.cms] Verify error:
Verification failure

140558031275680:error:0407006A:rsa
routines:RSA_padding_check_PKCS1_type_1:block type is not 01:rsa_pk1.c:100:
140558031275680:error:04067072:rsa routines:RSA_EAY_PUBLIC_DECRYPT:padding
check failed:rsa_eay.c:721:
140558031275680:error:2E09A09E:CMS
routines:CMS_SignerInfo_verify_content:verification failure:cms_sd.c:900:
140558031275680:error:2E09D06D:CMS routines:CMS_verify:content verify
error:cms_smime.c:425:

2013-04-30 17:43:07    DEBUG [keystoneclient.middleware.auth_token] Token
validation failure.
Traceback (most recent call last):
  File
"/usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py",
line 688, in _validate_user_token
    verified = self.verify_signed_token(user_token)
  File
"/usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py",
line 1043, in verify_signed_token
    if self.is_signed_token_revoked(signed_text):
  File
"/usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py",
line 1007, in is_signed_token_revoked
    revocation_list = self.token_revocation_list
  File
"/usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py",
line 1079, in token_revocation_list
    self.token_revocation_list = self.fetch_revocation_list()
  File
"/usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py",
line 1109, in fetch_revocation_list
    return self.cms_verify(data['signed'])
  File
"/usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py",
line 1038, in cms_verify
    raise err
CalledProcessError: Command 'openssl' returned non-zero exit status 4
2013-04-30 17:43:07    DEBUG [keystoneclient.middleware.auth_token] Marking
token MIIMbwYJKoZIhvcNA ... YAUt8D2KYQw== as unauthorized in memcache
2013-04-30 17:43:07  WARNING [keystoneclient.middleware.auth_token]
Authorization failed for token MIIMbwYJKoZIhvcNA ... YAUt8D2KYQw==
2013-04-30 17:43:07     INFO [keystoneclient.middleware.auth_token] Invalid
user token - rejecting request

MAYBE... somehow HAproxy is changing something in the header but i don't
think so. This is the haproxy configuration for the cinder API:

listen nova-api-cinder 172.19.136.1:8776
        balance  roundrobin
        option  tcplog
        server  heladera 172.19.136.245:8776  check

I don't understand why is the Verification Failure, and why i have openssl
involve in my authentication, I didn't change anything in the cinder
api-paste.ini file, besides the auth_host and service_host.

2013/4/30 Juan José Pavlik Salles <jjpavlik at gmail.com>

> Hi Jay, you are right, i'm trying to balance API calls with HAProxy. I
> installed HAproxy on 172.19.136.1 and configured all the openstack services
> to make the calls to that IP, then i use HAproxy to redirect the API calls
> to the real API servers (172.19.136.10 and 172.19.136.11), this is my
> configuration:
>
> I've these 4 nodes:
>
> 172.19.136.245:
> -Cinder
>
> 172.19.136.10:
> -Keystone
> -Glance (glance, api, registry)
> -Nova (compute, scheduler, etc)
>
> 172.19.136.11:
> -Keystone
> -Glance (glance, api, registry)
> -Nova (compute, scheduler, etc)
>
> 172.19.136.2 / 172.19.136.1:
> -Quantum server
> -RabbitMQ
> -MySQL
> -HAProxy (Listening on 172.19.136.1 for all the API calls, and balancing
> them to either 172.19.136.10 or 172.19.136.11, it also listens for cinder
> api calls and redirects them to 172.19.136.245)
>
> I didn't change all the endpoints yet, but all of them should redirect to
> 172.19.136.1, maybe that's the problem. What do you think?
>
> This configuration might look odd or strange, but i'm trying to build a
> redundant and scalable cloud (like in this article
> http://www.mirantis.com/blog/software-high-availability-load-balancing-openstack-cloud-api-servic/).
> Thanks!!!
>
>
> 2013/4/30 Jay Pipes <jaypipes at gmail.com>
>
>> On 04/29/2013 04:56 PM, Juan José Pavlik Salles wrote:
>> > Hi, i have spent the last days trying to solve this problem. I can't
>> > list my cinder volumes from my shell:
>> >
>> > root at locro:~# cinder --os-username=admin --os-tenant-name=admin
>> > --os-password=XXX --os-auth-url=http://172.19.136.1:35357/v2.0 --debug
>> list
>> >
>> > REQ: curl -i http://172.19.136.1:35357/v2.0/tokens -X POST -H
>> > "Content-Type: application/json" -H "Accept: application/json" -H
>> > "User-Agent: python-cinderclient" -d '{"auth": {"tenantName": "admin",
>> > "passwordCredentials": {"username": "admin", "password": "zGp05Nsa"}}}'
>> >
>> > RESP: [200] {'date': 'Mon, 29 Apr 2013 17:24:44 GMT', 'content-type':
>> > 'application/json', 'content-length': '7096', 'vary': 'X-Auth-Token'}
>> > RESP BODY: {"access": {"token": {"issued_at":
>> > "2013-04-29T17:24:44.044013", "expires": "2013-04-30T17:24:43Z", "id":
>> > "MIIMaQYJKoZIhvcNAQcC...", "tenant": {"description": null, "enabled":
>> > true, "id": "6aa3bf1ab68040218873a782f90cffa7", "name": "admin"}},
>> > "serviceCatalog": [{"endpoints": [{"adminURL":
>> > "http://172.19.136.11:8774/v2/6aa3bf1ab68040218873a782f90cffa7",
>> > "region": "RegionOne", "internalURL":
>> > "http://172.19.136.10:8774/v2/6aa3bf1ab68040218873a782f90cffa7", "id":
>> > "26178391275a42cfa3b786ab151c8f8a", "publicURL":
>> > "http://172.19.136.11:8774/v2/6aa3bf1ab68040218873a782f90cffa7"}],
>> > "endpoints_links": [], "type": "compute", "name": "nova"}, {"endpoints":
>> > [{"adminURL": "http://172.19.136.11:9696/", "region": "RegionOne",
>> > "internalURL": "http://172.19.136.11:9696/", "id":
>> > "1d0f394d83804ecaaa5ba708ccf0417b", "publicURL":
>> > "http://172.19.136.11:9696/"}], "endpoints_links": [], "type":
>> > "network", "name": "quantum"}, {"endpoints": [{"adminURL":
>> > "http://172.19.136.10:9292/v2", "region": "RegionOne", "internalURL":
>> > "http://172.19.136.11:9292/v2", "id":
>> > "11f37a313bad47f28b846cb9b94d458c", "publicURL":
>> > "http://172.19.136.11:9292/v2"}], "endpoints_links": [], "type":
>> > "image", "name": "glance"}, {"endpoints": [{"adminURL":
>> > "http://172.19.136.1:8776/v1/6aa3bf1ab68040218873a782f90cffa7",
>> > "region": "RegionOne", "internalURL":
>> > "http://172.19.136.1:8776/v1/6aa3bf1ab68040218873a782f90cffa7", "id":
>> > "1ebe70478edd45d087263a4dc457f03a", "publicURL":
>> > "http://172.19.136.1:8776/v1/6aa3bf1ab68040218873a782f90cffa7"}],
>> > "endpoints_links": [], "type": "volume", "name": "cinder"},
>> > {"endpoints": [{"adminURL": "http://172.19.136.11:8773/services/Admin",
>> > "region": "RegionOne", "internalURL":
>> > "http://172.19.136.10:8773/services/Cloud", "id":
>> > "4fd5bcbee3584c2b883b08f22f81de54", "publicURL":
>> > "http://172.19.136.10:8773/services/Cloud"}], "endpoints_links": [],
>> > "type": "ec2", "name": "ec2"}, {"endpoints": [{"adminURL":
>> > "http://172.19.136.10:8080/v1", "region": "RegionOne", "internalURL":
>> > "http://172.19.136.11:8080/v1/AUTH_6aa3bf1ab68040218873a782f90cffa7",
>> > "id": "65911114c36341a19006c328c6d0a2ae", "publicURL":
>> > "http://172.19.136.10:8080/v1/AUTH_6aa3bf1ab68040218873a782f90cffa7"}],
>> > "endpoints_links": [], "type": "object-store", "name": "swift"},
>> > {"endpoints": [{"adminURL": "http://172.19.136.11:35357/v2.0",
>> "region":
>> > "RegionOne", "internalURL": "http://172.19.136.10:5000/v2.0", "id":
>> > "0f9389d0485e4f2f9f7874c41181bd28", "publicURL":
>> > "http://172.19.136.10:5000/v2.0"}], "endpoints_links": [], "type":
>> > "identity", "name": "keystone"}], "user": {"username": "admin",
>> > "roles_links": [], "id": "3f82673b5fe0411ab5fd8216bdb693c6", "roles":
>> > [{"name": "KeystoneServiceAdmin"}, {"name": "KeystoneAdmin"}, {"name":
>> > "admin"}], "name": "admin"}, "metadata": {"is_admin": 0, "roles":
>> > ["6666fa99078a4f07a070e7e858c32f02", "36bba9ef0178448c8a654b75feb3a0f4",
>> > "a25581dd3470460b91ecaa29eca7205c"]}}}
>> >
>> > REQ: curl -i
>> >
>> http://172.19.136.1:8776/v1/6aa3bf1ab68040218873a782f90cffa7/volumes/detail
>> > -X GET -H "X-Auth-Project-Id: admin" -H "User-Agent:
>> > python-cinderclient" -H "Accept: application/json" -H "X-Auth-Token:
>> > MIIMaQYJKoZIhvcNAQcCo..."
>> >
>> > RESP: [401] {'date': 'Mon, 29 Apr 2013 17:24:44 GMT', 'content-length':
>> > '276', 'content-type': 'text/plain; charset=UTF-8', 'www-authenticate':
>> > "Keystone uri='http://172.19.136.1:35357'"}
>> > RESP BODY: 401 Unauthorized
>>
>> From the above, the authentication URI that you are supplying to
>> cinderclient is http://172.19.136.1:35357, which is not the same as what
>> is returned in the service catalog above, which has the internalURL for
>> the identity endpoint as http://172.19.136.10:5000/v2.0.
>>
>> Is this intended?
>>
>> -jay
>>
>>
>> _______________________________________________
>> OpenStack-operators mailing list
>> OpenStack-operators at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>>
>
>
>
> --
> Pavlik Juan José
>

-- 
Pavlik Juan José
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20130430/98731bf8/attachment.html>

Open Stack

[Openstack-operators] Authentication problems with cinder

OpenStack

Community

Documentation

Branding & Legal