<div dir="ltr">Hi Salman,<div><br></div><div>This has turned into a bit of fun -- I'm seeing a lot of wacky things. </div><div><br></div><div>First, I'm pretty this issue is local to multi-regions and doesn't have to do with having both Cinder v1 and v2 in the Keystone catalog. I changed my catalog to only have Cinder v2 and I still see multi-region issues. If I had more time, I would figure out how to forcefully make nova-api use Cinder v1 to confirm it's not a v2 issue, but I'm pretty confident that it is not.</div><div><br></div><div>Second, strange things happen depending on the UUID of the endpoints. Let's say I create two cinder v2 regions:</div><div><br></div><div><div>openstack endpoint create --region RegionOne volumev2 --publicurl <a href="http://10.1.0.112:8776/v2/%\(tenant_id\)s">http://10.1.0.112:8776/v2/%\(tenant_id\)s</a> --internalurl <a href="http://10.1.0.112:8776/v2/%\(tenant_id\)s">http://10.1.0.112:8776/v2/%\(tenant_id\)s</a> --adminurl <a href="http://10.1.0.112:8776/v2/%\(tenant_id\)s">http://10.1.0.112:8776/v2/%\(tenant_id\)s</a><br></div></div><div><br></div><div><div>+--------------+-----------------------------------------+</div><div>| Field        | Value                                   |</div><div>+--------------+-----------------------------------------+</div><div>| adminurl     | <a href="http://10.1.0.112:8776/v2/%(tenant_id)s">http://10.1.0.112:8776/v2/%(tenant_id)s</a> |</div><div>| id           | a46a5f86b0134944b66c25a7802f7b32        |</div><div>| internalurl  | <a href="http://10.1.0.112:8776/v2/%(tenant_id)s">http://10.1.0.112:8776/v2/%(tenant_id)s</a> |</div><div>| publicurl    | <a href="http://10.1.0.112:8776/v2/%(tenant_id)s">http://10.1.0.112:8776/v2/%(tenant_id)s</a> |</div><div>| region       | RegionOne                               |</div><div>| service_id   | 9cb6eeba4ae5484080ef1a5272b03367        |</div><div>| service_name | cinderv2                                |</div><div>| service_type | volumev2                                |</div><div>+--------------+-----------------------------------------+</div></div><div><br></div><div><div>openstack endpoint create --region RegionTwo volumev2 --publicurl <a href="http://10.1.0.113:8776/v2/%\(tenant_id\)s">http://10.1.0.113:8776/v2/%\(tenant_id\)s</a> --internalurl <a href="http://10.1.0.113:8776/v2/%\(tenant_id\)s">http://10.1.0.113:8776/v2/%\(tenant_id\)s</a> --adminurl <a href="http://10.1.0.113:8776/v2/%\(tenant_id\)s">http://10.1.0.113:8776/v2/%\(tenant_id\)s</a></div><div><br></div><div><div><div>+--------------+-----------------------------------------+</div><div>| Field        | Value                                   |</div><div>+--------------+-----------------------------------------+</div><div>| adminurl     | <a href="http://10.1.0.113:8776/v2/%(tenant_id)s">http://10.1.0.113:8776/v2/%(tenant_id)s</a> |</div><div>| id           | e9cd7a3fd8734b12a77154d73990261d        |</div><div>| internalurl  | <a href="http://10.1.0.113:8776/v2/%(tenant_id)s">http://10.1.0.113:8776/v2/%(tenant_id)s</a> |</div><div>| publicurl    | <a href="http://10.1.0.113:8776/v2/%(tenant_id)s">http://10.1.0.113:8776/v2/%(tenant_id)s</a> |</div><div>| region       | RegionTwo                               |</div><div>| service_id   | 9cb6eeba4ae5484080ef1a5272b03367        |</div><div>| service_name | cinderv2                                |</div><div>| service_type | volumev2                                |</div><div>+--------------+-----------------------------------------+</div></div><div><br></div></div></div><div>Note the IDs: RegionOne has a46... and RegionTwo has e9c...</div><div><br></div><div>When I do "openstack endpoint show cinderv2", RegionTwo will be displayed. Why? Because "e" comes after "a". If I regenerate the endpoints until RegionOne has a UUID that is alphabetically later than RegionTwo, then RegionOne will be displayed.</div><div><br></div><div>It does not matter what my OS_REGION_NAME environment variable is set to, either.</div><div><br></div><div>Now, let's say my nova.conf file for the nova-api server in RegionOne does not have the</div><div><br></div><div>[cinder]</div><div>os_region_name = RegionOne</div><div><br></div><div>setting I mentioned earlier (also, note that this needs to go in a "[cinder]" section).</div><div><br></div><div>If RegionOne's UUID comes earlier in the alphabet (so RegionOne would NOT be displayed in "openstack endpoint show cinderv2"), then "nova volume-attach" works. nova-api will grab the correct endpoint for RegionOne.</div><div><br></div><div>If RegionOne's UUID comes later in the alphabet, then "nova volume-attach" will try to contact RegionTwo. </div><div><br></div><div>My guess is that the openstack client is reporting the last returned catalog entry while the nova-api service is using the first returned catalog entry.</div><div><br></div><div>BUT, if I set </div><div><br></div><div><div>[cinder]</div><div>os_region_name = RegionOne</div></div><div><br></div><div>then "nova volume-attach" works, no matter what order the entries are in the catalog.</div><div><br></div><div><br></div><div>Given all of the above, here is what is working for me:</div><div><br></div><div>1. I have two services in my Keystone catalog:</div><div><br></div><div>cinder: volume</div><div>cinderv2: volumev2</div><div><br></div><div>It looks like you have 4:</div><div><br></div><div>cinder: volume</div><div>cinderv2: volumev2</div><div><span style="font-size:13px">cinder-hpc2n: volume</span><br></div><div><span style="font-size:13px">cinderv2-hpc2n: volumev2</span><span style="font-size:13px"><br></span></div><div><span style="font-size:13px"><br></span></div><div><span style="font-size:13px">I don't know if that makes a difference, but maybe simplify the services to just the standard 2. </span></div><div><br></div><div>2. nova.conf on the nova-api server has the "os_region_name" setting set to the correct local region. I only had to restart the nova-api service to see changes take effect, but maybe restart nova-* -- who knows at this point!</div><div><br></div><div>3. Check the spelling of your regions. I noticed that you sometimes have "regionOne" and other times have "RegionOne" but maybe that is just a typo from sanitizing your data.</div><div><br></div><div>Let me know if this helps :)</div><div><br></div><div>Joe</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Dec 10, 2015 at 2:31 AM, Salman Toor <span dir="ltr"><<a href="mailto:salman.toor@it.uu.se" target="_blank">salman.toor@it.uu.se</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">



<div style="word-wrap:break-word">
Hi Joe, 
<div><br>
</div>
<div>Thanks for your reply, yes Me and Marco are working together. </div>
<div><br>
</div>
<div>I have tried to used </div>
<div><br>
</div>
<div>os_region_name = RegionOne</div>
<div><br>
</div>
<div>both in cinder.conf and nova.conf but no luck so far. But the error message is a bit different. It seems like now the system is recognising the region. </div>
<div><br>
</div>
<div><br>
</div>
<div>
<p style="font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
<font color="#e32400">2015-12-10 09:32:54.661 26792 TRACE nova.api.openstack EndpointNotFound: internalURL endpoint for volume service named cinder in regionOne region not found</font></p>
<div><br>
</div>
</div>
<div>Following are my endpoints:</div>
<div><br>
</div>
<div>——</div>
<div>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
<b>[root@controller: ~]</b> # openstack endpoint show cinder</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
+--------------+------------------------------------------------+</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| Field        | Value                                          |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
+--------------+------------------------------------------------+</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| adminurl     | <a href="http://cinder:8776/v1/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">http://cinder:8776/v1/%(tenant_id)s</a>            |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| enabled      | True                                           |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| id           | e2693bcaf3da4be2810f04acd7995d7f               |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| internalurl  | <a href="http://cinder:8776/v1/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">http://cinder:8776/v1/%(tenant_id)s</a>            |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| publicurl    | <a href="http://smog.uppmax.uu.se:8776/v1/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">http://</a><a href="http://cinder:8776/v1/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">cinder</a><a href="http://smog.uppmax.uu.se:8776/v1/%(tenant_id)s" style="font-size:12.8px;color:rgb(17,85,204)" target="_blank">:8776/v1/%(tenant_id)s</a><span style="font-size:12.8px"> </span><span style="font-size:12.8px">|</span></p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| region       | regionOne                                      |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| service_id   | 7bd5d667ec4b4d65b6c1b0de8b303fe3               |</p><span class="">
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| service_name | cinder                                         |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| service_type | volume                                         |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
+--------------+------------------------------------------------+</p>
</span><p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
<b>[root@controller: ~]</b> # openstack endpoint show cinderv2</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
+--------------+------------------------------------------------+</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| Field        | Value                                          |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
+--------------+------------------------------------------------+</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| adminurl     | <a href="http://cinder:8776/v2/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">http://cinder:8776/v2/%(tenant_id)s</a>            |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| enabled      | True                                           |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| id           | 3000cb23c6ab4ee5b68876ee08257338               |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| internalurl  | <a href="http://cinder:8776/v2/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">http://cinder:8776/v2/%(tenant_id)s</a>            |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| publicurl    | <a href="http://smog.uppmax.uu.se:8776/v2/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">http://</a><a href="http://cinder:8776/v1/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">cinder</a><a href="http://smog.uppmax.uu.se:8776/v2/%(tenant_id)s" style="font-size:12.8px;color:rgb(17,85,204)" target="_blank">:8776/v2/%(tenant_id)s</a><span style="font-size:12.8px"> </span><span style="font-size:12.8px">|</span></p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| region       | regionOne                                      |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| service_id   | 61d8baeb4ee74c7798a60758b2f4171f               |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| service_name | cinderv2                                       |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| service_type | volumev2                                       |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
+--------------+------------------------------------------------+</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
<b>[root@controller: ~]</b> # openstack endpoint show cinder-hpc2n</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
+--------------+--------------------------------------------------+</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| Field        | Value                                            |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
+--------------+--------------------------------------------------+</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| adminurl     | <a href="http://u-mn01.hpc2n.umu.se:8776/v1/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">http://</a><a href="http://cinder:8776/v1/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">cinder</a><a href="http://u-mn01.hpc2n.umu.se:8776/v1/%(tenant_id)s" style="font-size:12.8px;color:rgb(17,85,204)" target="_blank">:8776/v1/%(tenant_id)s</a><span style="font-size:12.8px"> </span><span style="font-size:12.8px">|</span></p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| enabled      | True                                             |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| id           | c088f1f9ca0248a08eb60d4ac8492150                 |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| internalurl  | <a href="http://u-mn01.hpc2n.umu.se:8776/v1/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">http://</a><a href="http://cinder:8776/v1/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">cinder</a><a href="http://u-mn01.hpc2n.umu.se:8776/v1/%(tenant_id)s" style="font-size:12.8px;color:rgb(17,85,204)" target="_blank">:8776/v1/%(tenant_id)s</a><span style="font-size:12.8px"> </span><span style="font-size:12.8px">|</span></p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| publicurl    | <a href="http://u-mn01.hpc2n.umu.se:8776/v1/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">http://</a><a href="http://cinder:8776/v1/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">cinder</a><a href="http://u-mn01.hpc2n.umu.se:8776/v1/%(tenant_id)s" style="font-size:12.8px;color:rgb(17,85,204)" target="_blank">:8776/v1/%(tenant_id)s</a><span style="font-size:12.8px"> </span><span style="font-size:12.8px">|</span></p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| region       | HPC2N                                            |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| service_id   | 473f3eab043f4130b30d9ee64fe43703                 |</p><span class="">
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| service_name | cinder-hpc2n                                     |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| service_type | volume                                           |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
+--------------+--------------------------------------------------+</p>
</span><p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
<b>[root@controller: ~]</b> # openstack endpoint show cinderv2-hpc2n</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
+--------------+--------------------------------------------------+</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| Field        | Value                                            |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
+--------------+--------------------------------------------------+</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| adminurl     | <a href="http://u-mn01.hpc2n.umu.se:8776/v2/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">http://</a><a href="http://cinder:8776/v1/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">cinder</a><a href="http://u-mn01.hpc2n.umu.se:8776/v2/%(tenant_id)s" style="font-size:12.8px;color:rgb(17,85,204)" target="_blank">:8776/v2/%(tenant_id)s</a><span style="font-size:12.8px"> </span><span style="font-size:12.8px">|</span></p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| enabled      | True                                             |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| id           | 985e5b8b068e4c3d99c12788236ee0fe                 |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| internalurl  | <a href="http://u-mn01.hpc2n.umu.se:8776/v2/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">http://</a><a href="http://cinder:8776/v1/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">cinder</a><a href="http://u-mn01.hpc2n.umu.se:8776/v2/%(tenant_id)s" style="font-size:12.8px;color:rgb(17,85,204)" target="_blank">:8776/v2/%(tenant_id)s</a><span style="font-size:12.8px"> </span><span style="font-size:12.8px">|</span></p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| publicurl    | <a href="http://u-mn01.hpc2n.umu.se:8776/v2/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">http://</a><a href="http://cinder:8776/v1/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">cinder</a><a href="http://u-mn01.hpc2n.umu.se:8776/v2/%(tenant_id)s" style="font-size:12.8px;color:rgb(17,85,204)" target="_blank">:8776/v2/%(tenant_id)s</a><span style="font-size:12.8px"> </span><span style="font-size:12.8px">|</span></p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| region       | HPC2N                                            |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| service_id   | 089186fad4d7444b95e74ea9d7769423                 |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| service_name | cinderv2-hpc2n                                   |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| service_type | volumev2                                         |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
<br>
</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
+--------------+--------------------------------------------------+</p>
</div>
<div>——</div>
<div><br>
</div>
<div>Again if I remove hpc2n everything started to work fine. Can you tell me what service I suppose to restart (if required)?</div>
<div><br>
</div>
<div>Regards..</div><span class="HOEnZb"><font color="#888888">
<div>Salman</div>
<div><br>
</div>
<div><br>
</div>
</font></span><div><span class=""><br>
<div>
<div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word">
PhD, Scientific Computing<br>
Researcher, IT Department,<br>
Uppsala University.<br>
Senior Cloud Architect, <br>
SNIC.<br>
Cloud Application Expert,<br>
UPPMAX.<br>
Visiting Researcher,<br>
Helsinki Institute of Physics (HIP).<br>
<a href="mailto:salman.toor@it.uu.se" target="_blank">salman.toor@it.uu.se</a><br>
<a href="http://www.it.uu.se/katalog/salto690" target="_blank">http://www.it.uu.se/katalog/salto690</a></div>
</div>
<br>
</span><div><div class="h5"><div>
<blockquote type="cite">
<div>On 09 Dec 2015, at 17:48, Joe Topjian <<a href="mailto:joe@topjian.net" target="_blank">joe@topjian.net</a>> wrote:</div>
<br>
<div>
<div dir="ltr">Hi Salman,
<div><br>
</div>
<div>Someone mentioned this same issue yesterday in relation to Terraform (maybe a colleague of yours?), so given the two occurrences, I thought I'd look into this.</div>
<div><br>
</div>
<div>I have a Liberty environment readily available, so I created a second set of volume and volumev2 endpoints for a fictional region. Everything worked as expected, so I started reviewing the config files and saw that /etc/cinder/cinder.conf had
 an option</div>
<div><br>
</div>
<div>[DEFAULT]</div>
<div>os_region_name = RegionOne</div>
<div><br>
</div>
<div>I commented that out, but things still worked.</div>
<div><br>
</div>
<div>Then in /etc/nova/nova.conf, I saw:</div>
<div><br>
</div>
<div>[cinder]</div>
<div>os_region_name = RegionOne</div>
<div><br>
</div>
<div>commenting this out caused volume attachments to hang indefinitely because nova was trying to contact cinder at RegionTwo (I'm assuming this is the first catalog entry that was returned).</div>
<div><br>
</div>
<div>Given this is a Liberty environment, it's not accurately reproducing your problem, but could you check and see if you have that option set in nova.conf?</div>
<div><br>
</div>
<div>I have a Kilo environment in the process of building. Once it has finished, I'll see if I can reproduce your error there.</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Joe</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Wed, Dec 9, 2015 at 4:35 AM, Salman Toor <span dir="ltr">
<<a href="mailto:salman.toor@it.uu.se" target="_blank">salman.toor@it.uu.se</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">
<div>Hi, </div>
<div><br>
</div>
<div>I am using Kilo release on CentOS. We have recently enabled multiple regions and it seems that Cinder have some problems with  multiple endpoints. </div>
<div><br>
</div>
<div>Thinks are working fine with nova but cinder is behaving strange. Here are my endpoints </div>
<div><br>
</div>
<div><br>
</div>
<div>————</div>
<div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
<div style="margin:0px;line-height:normal"><span style="color:#5330e1"><b>[root@controller: ~]</b></span> # openstack service list</div>
<div style="margin:0px;line-height:normal">+----------------------------------+----------------+--------------+</div>
<div style="margin:0px;line-height:normal">| ID                               | Name           | Type         |</div>
<div style="margin:0px;line-height:normal">+----------------------------------+----------------+--------------+</div>
<div style="margin:0px;line-height:normal"><font color="#e32400">| 0a33e6f259794ff2a99e626be37c0c2b | cinderv2-hpc2n | volumev2     |</font></div>
<div style="margin:0px;line-height:normal">| 1fcae9bd76304853a3168c39c7fe8e6b | nova           | compute      |</div>
<div style="margin:0px;line-height:normal">| 2c7828120c294d3f82e3a17835babb85 | neutron        | network      |</div>
<div style="margin:0px;line-height:normal">| 3804fcd8f9494d30b589b55fe6abb811 | nova-hpc2n     | compute      |</div>
<div style="margin:0px;line-height:normal">| 478eff4e96464ae8a958ba29f750b14c | glance         | image        |</div>
<div style="margin:0px;line-height:normal"><font color="#e32400">| 4a5a771d915e43c28e66538b8bc6e625 | cinder         | volume       |</font></div>
<div style="margin:0px;line-height:normal"><font color="#e32400">| 72d1be82b2e5478dbf0f3fb9e7ba969d | cinderv2       | volumev2     |</font></div>
<div style="margin:0px;line-height:normal">| 97f977f8a7a04bae89da167fd25dc06c | glance-hpc2n   | image        |</div>
<div style="margin:0px;line-height:normal"><font color="#e32400">| a985795b49e2440db82970b81248c86e | cinder-hpc2n   | volume       |</font></div>
<div style="margin:0px;line-height:normal">| dccd39b92ab547ddaf9047b38620145a | swift          | object-store |</div>
<div style="margin:0px;line-height:normal">| ebb1660d1d9746759a48de921521bfad | keystone       | identity     |</div>
<div style="margin:0px;line-height:normal">+----------------------------------+----------------+--------------+</div>
</div>
</div>
<div><br>
</div>
<div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
<span style="color:#5330e1"><b>[root@controller: ~]</b></span> # openstack endpoint show a985795b49e2440db82970b81248c86e</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
+--------------+--------------------------------------------------+</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
| Field        | Value                                            |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
+--------------+--------------------------------------------------+</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
| adminurl     | http://<URL>:8776/v1/%(tenant_id)s |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
| enabled      | True                                             |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
| id           | d4003e91ddf24cfb9fa497da81b01a18                 |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
| internalurl  | http://<URL>:8776/v1/%(tenant_id)s |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
| publicurl    | http://<URL>:8776/v1/%(tenant_id)s |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
| region       | HPC2N                                            |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
| service_id   | a985795b49e2440db82970b81248c86e                 |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
| service_name | cinder-hpc2n                                     |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
| service_type | volume                                           |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
+--------------+--------------------------------------------------+</div>
</div>
<div><br>
</div>
<div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
<span style="color:#5330e1"><b>[root@controller: ~]</b></span> # openstack endpoint show 4a5a771d915e43c28e66538b8bc6e625</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
+--------------+------------------------------------------------+</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
| Field        | Value                                          |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
+--------------+------------------------------------------------+</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
| adminurl     | http://<URL>:8776/v1/%(tenant_id)s            |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
| enabled      | True                                           |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
| id           | 5f19c0b535674dbd9e318c7b6d61b3bc               |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
| internalurl  | http://<URL>:8776/v1/%(tenant_id)s            |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
| publicurl    | http://<URL>:8776/v1/%(tenant_id)s |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
| region       | regionOne                                      |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
| service_id   | 4a5a771d915e43c28e66538b8bc6e625               |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
| service_name | cinder                                         |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
| service_type | volume                                         |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
+--------------+------------------------------------------------+</div>
</div>
<div><br>
</div>
<div>And same for v2 endpoints  </div>
<div><br>
</div>
<div>————</div>
<div><br>
</div>
<div>——— nova-api.log ———</div>
<div><br>
</div>
<div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
achmentController object at 0x598a3d0>>, body: {"volumeAttachment": {"device": "", "volumeId": "93d96eab-e3fd-4131-9549-ed51e7299da2"}} _process_stack /usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py:780</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.847 3376 INFO nova.api.openstack.compute.contrib.volumes [req-d6e1380d-c6bc-4911-b2c6-251bc8b4c62c a62c20fdf99c443a924f0d50a51514b1 3c9d997982e04c6db0e02b82fa18fdd8 - - -] Attach volume 93d96eab-e3fd-4131-9549-ed51e7299da2 to instance 3a4c8722-52a7-48f2-beb7-db8938698a0d
 at </div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.847 3376 DEBUG nova.compute.api [req-d6e1380d-c6bc-4911-b2c6-251bc8b4c62c a62c20fdf99c443a924f0d50a51514b1 3c9d997982e04c6db0e02b82fa18fdd8 - - -] [instance: 3a4c8722-52a7-48f2-beb7-db8938698a0d] Fetching instance by UUID get /usr/lib/python2.7/site-packages/nova/compute/api.py:1911</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 ERROR nova.api.openstack [req-d6e1380d-c6bc-4911-b2c6-251bc8b4c62c a62c20fdf99c443a924f0d50a51514b1 3c9d997982e04c6db0e02b82fa18fdd8 - - -] Caught error: internalURL endpoint for volume service named cinder not found</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack Traceback (most recent call last):</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/api/openstack/__init__.py", line 125, in __call__</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     return req.get_response(self.application)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1296, in send</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     application, catch_exc_info=False)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1260, in call_application</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     app_iter = application(self.environ, start_response)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 144, in __call__</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     return resp(environ, start_response)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 634, in __call__</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     return self._call_app(env, start_response)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 554, in _call_app</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     return self._app(env, _fake_start_response)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 144, in __call__</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     return resp(environ, start_response)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 144, in __call__</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     return resp(environ, start_response)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/routes/middleware.py", line 131, in __call__</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     response = self.app(environ, start_response)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 144, in __call__</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     return resp(environ, start_response)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 130, in __call__</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     resp = self.call_func(req, *args, **self.kwargs)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 195, in call_func</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     return self.func(req, *args, **kwargs)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py", line 756, in __call__</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     content_type, body, accept)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py", line 821, in _process_stack</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     action_result = self.dispatch(meth, request, action_args)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py", line 911, in dispatch</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     return method(req=request, **action_args)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/contrib/volumes.py", line 305, in create</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     volume_id, device)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 230, in wrapped</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     return func(self, context, target, *args, **kwargs)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 219, in inner</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     return function(self, context, instance, *args, **kwargs)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 200, in inner</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     return f(self, context, instance, *args, **kw)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 3035, in attach_volume</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     disk_bus, device_type)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 3016, in _attach_volume</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     volume_bdm.destroy()</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     six.reraise(self.type_, self.value, self.tb)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 3009, in _attach_volume</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     volume = self.volume_api.get(context, volume_id)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/volume/cinder.py", line 214, in wrapper</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     res = method(self, ctx, volume_id, *args, **kwargs)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/volume/cinder.py", line 286, in get</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     item = cinderclient(context).volumes.get(volume_id)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/volume/cinder.py", line 115, in cinderclient</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     url = _SESSION.get_endpoint(auth, **service_parameters)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 635, in get_endpoint</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     return auth.get_endpoint(self, **kwargs)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystoneclient/auth/identity/base.py", line 219, in get_endpoint</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     service_name=service_name)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystoneclient/utils.py", line 318, in inner</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     return func(*args, **kwargs)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystoneclient/service_catalog.py", line 254, in url_for</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     raise exceptions.EndpointNotFound(msg)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
<font color="#ff4013">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack EndpointNotFound: internalURL endpoint for volume service named cinder not found</font></div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack</div>
</div>
<div><br>
</div>
<div>———</div>
<div><br>
</div>
<div>If I remove cinder-hpc2n and cinderv2-hpc2n from the service list, everything started to work fine. </div>
<div><br>
</div>
<div>This is how I have created endpoints: </div>
<div><br>
</div>
<div><br>
</div>
<div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
openstack endpoint create --publicurl http://<URL>:8776/v1/%\(tenant_id\)s --internalurl http://<URL>:8776/v1/%\(tenant_id\)s --adminurl http://<URL>:8776/v1/%\(tenant_id\)s --region regionOne 4a5a771d915e43c28e66538b8bc6e625 </div>
</div>
<div><br>
</div>
<div><span style="font-family:Menlo;font-size:11px">openstack endpoint create --publicurl http://<URL>:8776/v1/%\(tenant_id\)s --internalurl http://<URL>:8776/v1/%\(tenant_id\)s --adminurl http://<URL>:8776/v1/%\(tenant_id\)s --region HPC2N </span><span style="font-family:Menlo;font-size:11px">a985795b49e2440db82970b81248c86e</span></div>
<div><br>
</div>
<div>using the service-id. </div>
<div><br>
</div>
<div><br>
</div>
<div>Anyone experiencing this behavior? or Any suggestion how to fix it? Again regions are working fine with nova and glance. </div>
<div><br>
</div>
<div><br>
</div>
<div>Regards..</div>
<div>Salman. </div>
<div><br>
</div>
<div><br>
</div>
<div>
<div style="letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word">
<div>PhD, Scientific Computing<br>
Researcher, IT Department,<br>
Uppsala University.</div>
<div>
<div style="word-wrap:break-word">Senior Cloud Architect, </div>
<div style="word-wrap:break-word">SNIC.</div>
Cloud Application Expert,<br>
UPPMAX.<br>
Visiting Researcher,<br>
Helsinki Institute of Physics (HIP).<br>
<a href="mailto:salman.toor@it.uu.se" target="_blank">salman.toor@it.uu.se</a><br>
<a href="http://www.it.uu.se/katalog/salto690" target="_blank">http://www.it.uu.se/katalog/salto690</a></div>
</div>
</div>
<br>
</div>
<br>
_______________________________________________<br>
OpenStack-operators mailing list<br>
<a href="mailto:OpenStack-operators@lists.openstack.org" target="_blank">OpenStack-operators@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
</div>
<br>
</div></div></div>
</div>

</blockquote></div><br></div>