<div dir="ltr">Hi Salman,<div><br></div><div>That's awesome news. Glad it's working. :)</div><div><br></div><div>Joe</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Dec 11, 2015 at 3:12 PM, Salman Toor <span dir="ltr"><<a href="mailto:salman.toor@it.uu.se" target="_blank">salman.toor@it.uu.se</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">




<div>
<div style="direction:ltr;font-family:Tahoma;color:#000000;font-size:10pt">Hi, 
<div><span style="font-size:10pt"><br>
</span></div>
<div><span style="font-size:10pt">It is working by setting exact names of the services. </span></div>
<div><span style="font-size:10pt"><br>
</span></div>
<div><span style="font-size:10pt"><br>
</span></div>
<p><span><b>[root@smog: ~]</b></span><span> # openstack service list </span></p>
<p><span>+----------------------------------+--------------+--------------+</span></p>
<p><span>| ID                               | Name         | Type         |</span></p>
<p><span>+----------------------------------+--------------+--------------+</span></p><span class="">
<p><span>| 1fcae9bd76304853a3168c39c7fe8e6b | nova         | compute      |</span></p>
<p><span>| 2c7828120c294d3f82e3a17835babb85 | neutron      | network      |</span></p>
<p><span>| 3804fcd8f9494d30b589b55fe6abb811 | nova-hpc2n   | compute      |</span></p>
<p><span>| 478eff4e96464ae8a958ba29f750b14c | glance       | image        |</span></p>
<p><span><font color="#ff0000">| 61d8baeb4ee74c7798a60758b2f4171f | cinderv2     | volumev2     |</font></span></p>
</span><p><span><font color="#ff0000">| 75f89962c7864507b07055fbfc98053e | cinder       | volume       |</font></span></p><span class="">
<p><span><font color="#ff0000">| 7bd5d667ec4b4d65b6c1b0de8b303fe3 | cinder       | volume       |</font></span></p>
<p><span>| 97f977f8a7a04bae89da167fd25dc06c | glance-hpc2n | image        |</span></p>
</span><p><span><font color="#ff0000">| 9d2a7ef6b36c45b096e552bf73cb89ae | cinderv2     | volumev2     |</font></span></p><span class="">
<p><span>| dccd39b92ab547ddaf9047b38620145a | swift        | object-store |</span></p>
<p><span>| ebb1660d1d9746759a48de921521bfad | keystone     | identity     |</span></p>
</span><p><span>+----------------------------------+--------------+--------------+</span></p>
<p><span><b>[root@smog: ~]</b></span><span> # openstack endpoint list </span></p>
<p><span>+----------------------------------+-----------+--------------+--------------+</span></p><span class="">
<p><span>| ID                               | Region    | Service Name | Service Type |</span></p>
</span><p><span>+----------------------------------+-----------+--------------+--------------+</span></p><span class="">
<p><span><font color="#ff0000">| 3000cb23c6ab4ee5b68876ee08257338 | regionOne | cinderv2     | volumev2     |</font></span></p>
<p><span>| e21e935ed861484e976d1b93e0fda0f0 | regionOne | nova         | compute      |</span></p>
</span><p><span>| cdfb008c47a0472ab3a93e6ee07e9ba4 | regionOne | neutron      | network      |</span></p>
<p><span><font color="#ff0000">| e2693bcaf3da4be2810f04acd7995d7f | regionOne | cinder       | volume       |</font></span></p>
<p><span><font color="#ff0000">| 1400c42cbf154f63b0c3e8d64352d1f2 | HPC2N     | cinderv2     | volumev2     |</font></span></p>
<p><span>| 04acd1666704433d991cf9a75957c815 | HPC2N     | glance-hpc2n | image        |</span></p><span class="">
<p><span>| cfd1c766c56744309767cd84034f9bfb | regionOne | swift        | object-store |</span></p>
<p><span>| abe46f0d86064056927a2177a705787c | HPC2N     | nova-hpc2n   | compute      |</span></p>
</span><p><span><font color="#ff0000">| 52bb09199ec84ef09642151348eab695 | HPC2N     | cinder       | volume       |</font></span></p><span class="">
<p><span>| ac522bcd576b4d2b9f5adfb5405730be | regionOne | keystone     | identity     |</span></p>
<p><span>| bad524f18fd74ec3b7fb6647ea661686 | regionOne | glance       | image        |</span></p>
</span><div><span style="font-size:10pt">+----------------------------------+-----------+--------------+--------------+</span><span style="font-size:10pt"> </span></div>
<div><br>
</div>
<div><br>
</div>
<div>But different names are working perfectly fine with glance and nova.  </div>
<div><br>
</div>
<div>Anyways thanks for all your time and efforts. If I manage to reproduce it on devstack, will report you back. </div>
<div><br>
</div>
<div><span style="font-size:10pt">Regards..</span></div>
<div>Salman </div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
<div style="font-family:Times New Roman;color:#000000;font-size:16px">
<hr>
<div style="direction:ltr"><font face="Tahoma" size="2" color="#000000"><b>From:</b> Joe Topjian [<a href="mailto:joe@topjian.net" target="_blank">joe@topjian.net</a>]<br>
<b>Sent:</b> Friday, December 11, 2015 7:16 PM<br>
<b>To:</b> Salman Toor<br>
<b>Cc:</b> <a href="mailto:openstack-operators@lists.openstack.org" target="_blank">openstack-operators@lists.openstack.org</a><br>
<b>Subject:</b> Re: [Openstack-operators] Cinder API with multiple regions not working.<br>
</font><br>
</div><div><div class="h5">
<div></div>
<div>
<div dir="ltr">Hi Salman,
<div><br>
</div>
<div>I'm stumped.</div>
<div><br>
</div>
<div>I was able to confirm that Keystone acts a little differently when you have multiple regional endpoints connected to the same "service_name" and "service_type" than if you have multiple regional endpoints, each with their own "service_name" but same "service_type"'s. </div>
<div><br>
</div>
<div>For example, you have:</div>
<div><br>
</div>
<div>
<div>regionOne, cinder, volume</div>
<div>regionOne, cinderv2, volumev2</div>
<div>HPC2N, cinderhpc2n, volume</div>
<div>HPC2N, cinderv2hpc2n, volumev2</div>
</div>
<div><br>
</div>
<div>When you do "openstack endpoint show volume", you should not get any output. But if you do "openstack endpoint show cinderhpc2n", then you will see output. Swap the service_type and service_names and the pattern continues.</div>
<div><br>
</div>
<div>However, I am still unable to reproduce your issue.</div>
<div><br>
</div>
<div>My initial tests used the following format:</div>
<div><br>
</div>
<div>
<div>
<div>regionOne, cinder, volume</div>
<div>regionOne, cinderv2, volumev2</div>
<div>HPC2N, cinder, volume</div>
<div>HPC2N, cinderv2, volumev2</div>
</div>
</div>
<div><br>
</div>
<div>With this format, you will run into the UUID ordering issue I described previously. But again, I'm not able to reproduce the error you're seeing.</div>
<div><br>
</div>
<div>I really hate to give this answer, but at this point, I'd recommend setting up a devstack environment and comparing the configuration files under /etc/nova and /etc/cinder. This problem is not occurring for me in devstack, so there has to be some difference
 between the two that is causing this issue.</div>
<div><br>
</div>
<div>Once you have devstack installed, you can create a fictional second region by doing:</div>
<div><br>
</div>
<div>Your current style:</div>
<div><br>
</div>
<div>
<div>openstack service create --name cinderhpc2n volume</div>
<div>openstack service create --name cinderhpc2nv2 volumev2</div>
<div>openstack endpoint create --region HPC2N cinderhpc2n --publicurl <a href="http://10.1.0.113:8776/v1/%\(tenant_id\)s" target="_blank">
http://10.1.0.113:8776/v1/%\(tenant_id\)s</a> --internalurl <a href="http://10.1.0.113:8776/v1/%\(tenant_id\)s" target="_blank">
http://10.1.0.113:8776/v1/%\(tenant_id\)s</a> --adminurl <a href="http://10.1.0.113:8776/v1/%\(tenant_id\)s" target="_blank">
http://10.1.0.113:8776/v1/%\(tenant_id\)s</a></div>
<div>openstack endpoint create --region HPC2N cinderhpc2nv2 --publicurl <a href="http://10.1.0.113:8776/v2/%\(tenant_id\)s" target="_blank">
http://10.1.0.113:8776/v2/%\(tenant_id\)s</a> --internalurl <a href="http://10.1.0.113:8776/v2/%\(tenant_id\)s" target="_blank">
http://10.1.0.113:8776/v2/%\(tenant_id\)s</a> --adminurl <a href="http://10.1.0.113:8776/v2/%\(tenant_id\)s" target="_blank">
http://10.1.0.113:8776/v2/%\(tenant_id\)s</a></div>
</div>
<div><br>
</div>
<div>My style which I was originally testing with:</div>
<div><br>
</div>
<div>
<div>openstack endpoint create --region HPC2N volume --publicurl <a href="http://10.1.0.113:8776/v1/%\(tenant_id\)s" target="_blank">
http://10.1.0.113:8776/v1/%\(tenant_id\)s</a> --internalurl <a href="http://10.1.0.113:8776/v1/%\(tenant_id\)s" target="_blank">
http://10.1.0.113:8776/v1/%\(tenant_id\)s</a> --adminurl <a href="http://10.1.0.113:8776/v1/%\(tenant_id\)s" target="_blank">
http://10.1.0.113:8776/v1/%\(tenant_id\)s</a></div>
<div>openstack endpoint create --region HPC2N volumev2 --publicurl <a href="http://10.1.0.113:8776/v2/%\(tenant_id\)s" target="_blank">
http://10.1.0.113:8776/v2/%\(tenant_id\)s</a> --internalurl <a href="http://10.1.0.113:8776/v2/%\(tenant_id\)s" target="_blank">
http://10.1.0.113:8776/v2/%\(tenant_id\)s</a> --adminurl <a href="http://10.1.0.113:8776/v2/%\(tenant_id\)s" target="_blank">
http://10.1.0.113:8776/v2/%\(tenant_id\)s</a></div>
</div>
<div><br>
</div>
<div>If the IP address is not in use, you will see commands hang when they try to reach the wrong region. I find this to be an easy troubleshooting method to discover where problems are being hit.</div>
<div><br>
</div>
<div>If you figure out what the issue is, please let me know as I'm very curious.</div>
<div><br>
</div>
<div>Joe</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Fri, Dec 11, 2015 at 3:58 AM, Salman Toor <span dir="ltr">
<<a href="mailto:salman.toor@it.uu.se" target="_blank">salman.toor@it.uu.se</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">Hi, 
<div><br>
</div>
<div>Still no luck. I have remove the “-“ from the names.</div>
<div><br>
</div>
<div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(83,48,225)">
<b>[root@controller: ~/openstack]</b><span style="color:rgb(0,0,0)"> # openstack service list </span></div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">+----------------------------------+---------------+--------------+</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| ID                               | Name          | Type         |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">+----------------------------------+---------------+--------------+</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><font color="#b51a00">| 0f6736013cba469b9ddf21988a071952 | cinderhpc2n   | volume       |</font></div>
<span>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| 1fcae9bd76304853a3168c39c7fe8e6b | nova          | compute      |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| 2c7828120c294d3f82e3a17835babb85 | neutron       | network      |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| 3804fcd8f9494d30b589b55fe6abb811 | nova-hpc2n    | compute      |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| 478eff4e96464ae8a958ba29f750b14c | glance        | image        |</div>
</span>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><font color="#b51a00">| 61d8baeb4ee74c7798a60758b2f4171f | cinderv2      | volumev2     |</font></div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><font color="#b51a00">| 7bd5d667ec4b4d65b6c1b0de8b303fe3 | cinder        | volume       |</font></div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| 97f977f8a7a04bae89da167fd25dc06c | glance-hpc2n  | image        |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| dccd39b92ab547ddaf9047b38620145a | swift         | object-store |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><font color="#b51a00">| dec231e13a114365b6e094421204255f | cinderhpc2nv2 | volumev2     |</font></div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| ebb1660d1d9746759a48de921521bfad | keystone      | identity     |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">+----------------------------------+---------------+--------------+</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(83,48,225)">
<b>[root@controller: ~/openstack]</b><span style="color:rgb(0,0,0)"> # openstack endpoint list</span></div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">+----------------------------------+-----------+---------------+--------------+</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| ID                               | Region    | Service Name  | Service Type |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">+----------------------------------+-----------+---------------+--------------+</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><font color="#e32400">| 3000cb23c6ab4ee5b68876ee08257338 | regionOne | cinderv2      | volumev2     |</font></div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| e21e935ed861484e976d1b93e0fda0f0 | regionOne | nova          | compute      |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><font color="#e32400">| 42211e6e977d43308a4e288e68b97263 | HPC2N     | cinderhpc2nv2 | volumev2     |</font></div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| cdfb008c47a0472ab3a93e6ee07e9ba4 | regionOne | neutron       | network      |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><font color="#e32400">| e2693bcaf3da4be2810f04acd7995d7f | regionOne | cinder        | volume       |</font></div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| 04acd1666704433d991cf9a75957c815 | HPC2N     | glance-hpc2n  | image        |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><font color="#b51a00">| eb1a12119c5c4b2dba78927982121be7 | HPC2N     | cinderhpc2n   | volume       |</font></div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| cfd1c766c56744309767cd84034f9bfb | regionOne | swift         | object-store |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| abe46f0d86064056927a2177a705787c | HPC2N     | nova-hpc2n    | compute      |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| ac522bcd576b4d2b9f5adfb5405730be | regionOne | keystone      | identity     |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| bad524f18fd74ec3b7fb6647ea661686 | regionOne | glance        | image        |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">+----------------------------------+-----------+---------------+--------------+</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><span style="color:rgb(83,48,225)"><b>[root@controller: ~/openstack]</b></span> # systemctl restart openstack-nova-api.service</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><br>
</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">Even with all the services didn’t help. </div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><br>
</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><span style="color:rgb(83,48,225)"><b>[root@controller: ~/openstack]</b></span> #<font color="#b51a00">
</font><font color="#e32400">nova volume-attach 800d8ba0-cc17-4877-894a-89adecfb5eb7 85ab8b8a-c75c-45a1-9f51-44ed75ba3210 /dev/vdb</font></div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><font color="#e32400">ERROR (ClientException): The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-f0d9372d-863b-43f6-8281-5c9196a866af)</font></div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><span style="color:rgb(83,48,225)"><b>[root@controller: ~/openstack]</b></span> #
<font color="#b51a00">openstack service delete cinderhpc2n</font></div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><span style="color:rgb(83,48,225)"><b>[root@controller: ~/openstack]</b></span> #<font color="#e32400"> openstack service delete cinderhpc2nv2</font></div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(83,48,225)">
<b>[root@controller: ~/openstack]</b><span style="color:rgb(0,0,0)"> # </span></div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(83,48,225)">
<b>[root@controller: ~/openstack]</b><span style="color:rgb(0,0,0)"> # </span></div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(83,48,225)">
<b>[root@controller: ~/openstack]</b><span style="color:rgb(0,0,0)"> # </span></div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(83,48,225)">
<b>[root@controller: ~/openstack]</b><span style="color:rgb(0,0,0)"> # </span></div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><span style="color:rgb(83,48,225)"><b>[root@controller: ~/openstack]</b></span> #
<font color="#e32400">nova volume-attach 800d8ba0-cc17-4877-894a-89adecfb5eb7 85ab8b8a-c75c-45a1-9f51-44ed75ba3210 /dev/vdb</font></div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">+----------+--------------------------------------+</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| Property | Value                                |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">+----------+--------------------------------------+</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| device   | /dev/vdb                             |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| id       | 85ab8b8a-c75c-45a1-9f51-44ed75ba3210 |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| serverId | 800d8ba0-cc17-4877-894a-89adecfb5eb7 |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| volumeId | 85ab8b8a-c75c-45a1-9f51-44ed75ba3210 |</div>
<p style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"></p>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">+----------+--------------------------------------+</div>
</div>
<div><br>
</div>
<div><br>
</div>
<div>—— nova-api.log ——</div>
<div><span style="font-family:Menlo;font-size:11px"><font color="#e32400">2015-12-11 09:43:08.534 16529 TRACE nova.api.openstack EndpointNotFound: internalURL endpoint for volume service named cinder in regionOne region not found</font></span></div>
<div>——</div>
<div><br>
</div>
<div>——nova.conf</div>
<div>[cinder]</div>
<div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">os_region_name = regionOne</div>
</div>
<div>——</div>
<div><br>
</div>
<div>—— cinder.conf</div>
<div>[default]</div>
<div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">os_region_name = regionOne</div>
</div>
<div>——</div>
<div><br>
</div>
<div>Regards..</div>
<span><font color="#888888">
<div>Salman. </div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
</font></span>
<div><span><br>
<div>
<div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word">
<div>PhD, Scientific Computing<br>
Researcher, IT Department,<br>
Uppsala University.</div>
<div>
<div style="word-wrap:break-word">Senior Cloud Architect, </div>
<div style="word-wrap:break-word">SNIC.</div>
Cloud Application Expert,<br>
UPPMAX.<br>
Visiting Researcher,<br>
Helsinki Institute of Physics (HIP).<br>
<a href="mailto:salman.toor@it.uu.se" target="_blank">salman.toor@it.uu.se</a><br>
<a href="http://www.it.uu.se/katalog/salto690" target="_blank">http://www.it.uu.se/katalog/salto690</a></div>
</div>
</div>
<br>
</span>
<div>
<div>
<div>
<blockquote type="cite">
<div>On 10 Dec 2015, at 23:35, Joe Topjian <<a href="mailto:joe@topjian.net" target="_blank">joe@topjian.net</a>> wrote:</div>
<br>
<div>
<div dir="ltr">Hi Salman,
<div><br>
</div>
<div>This has turned into a bit of fun -- I'm seeing a lot of wacky things. </div>
<div><br>
</div>
<div>First, I'm pretty this issue is local to multi-regions and doesn't have to do with having both Cinder v1 and v2 in the Keystone catalog. I changed my catalog to only have Cinder v2 and I still see multi-region issues. If I had more time, I would figure
 out how to forcefully make nova-api use Cinder v1 to confirm it's not a v2 issue, but I'm pretty confident that it is not.</div>
<div><br>
</div>
<div>Second, strange things happen depending on the UUID of the endpoints. Let's say I create two cinder v2 regions:</div>
<div><br>
</div>
<div>
<div>openstack endpoint create --region RegionOne volumev2 --publicurl <a href="http://10.1.0.112:8776/v2/%/(tenant_id/)s" target="_blank">
http://10.1.0.112:8776/v2/%\(tenant_id\)s</a> --internalurl <a href="http://10.1.0.112:8776/v2/%/(tenant_id/)s" target="_blank">
http://10.1.0.112:8776/v2/%\(tenant_id\)s</a> --adminurl <a href="http://10.1.0.112:8776/v2/%/(tenant_id/)s" target="_blank">
http://10.1.0.112:8776/v2/%\(tenant_id\)s</a><br>
</div>
</div>
<div><br>
</div>
<div>
<div>+--------------+-----------------------------------------+</div>
<div>| Field        | Value                                   |</div>
<div>+--------------+-----------------------------------------+</div>
<div>| adminurl     | <a href="http://10.1.0.112:8776/v2/%(tenant_id)s" target="_blank">
http://10.1.0.112:8776/v2/%(tenant_id)s</a> |</div>
<div>| id           | a46a5f86b0134944b66c25a7802f7b32        |</div>
<div>| internalurl  | <a href="http://10.1.0.112:8776/v2/%(tenant_id)s" target="_blank">
http://10.1.0.112:8776/v2/%(tenant_id)s</a> |</div>
<div>| publicurl    | <a href="http://10.1.0.112:8776/v2/%(tenant_id)s" target="_blank">
http://10.1.0.112:8776/v2/%(tenant_id)s</a> |</div>
<div>| region       | RegionOne                               |</div>
<div>| service_id   | 9cb6eeba4ae5484080ef1a5272b03367        |</div>
<div>| service_name | cinderv2                                |</div>
<div>| service_type | volumev2                                |</div>
<div>+--------------+-----------------------------------------+</div>
</div>
<div><br>
</div>
<div>
<div>openstack endpoint create --region RegionTwo volumev2 --publicurl <a href="http://10.1.0.113:8776/v2/%/(tenant_id/)s" target="_blank">
http://10.1.0.113:8776/v2/%\(tenant_id\)s</a> --internalurl <a href="http://10.1.0.113:8776/v2/%/(tenant_id/)s" target="_blank">
http://10.1.0.113:8776/v2/%\(tenant_id\)s</a> --adminurl <a href="http://10.1.0.113:8776/v2/%/(tenant_id/)s" target="_blank">
http://10.1.0.113:8776/v2/%\(tenant_id\)s</a></div>
<div><br>
</div>
<div>
<div>
<div>+--------------+-----------------------------------------+</div>
<div>| Field        | Value                                   |</div>
<div>+--------------+-----------------------------------------+</div>
<div>| adminurl     | <a href="http://10.1.0.113:8776/v2/%(tenant_id)s" target="_blank">
http://10.1.0.113:8776/v2/%(tenant_id)s</a> |</div>
<div>| id           | e9cd7a3fd8734b12a77154d73990261d        |</div>
<div>| internalurl  | <a href="http://10.1.0.113:8776/v2/%(tenant_id)s" target="_blank">
http://10.1.0.113:8776/v2/%(tenant_id)s</a> |</div>
<div>| publicurl    | <a href="http://10.1.0.113:8776/v2/%(tenant_id)s" target="_blank">
http://10.1.0.113:8776/v2/%(tenant_id)s</a> |</div>
<div>| region       | RegionTwo                               |</div>
<div>| service_id   | 9cb6eeba4ae5484080ef1a5272b03367        |</div>
<div>| service_name | cinderv2                                |</div>
<div>| service_type | volumev2                                |</div>
<div>+--------------+-----------------------------------------+</div>
</div>
<div><br>
</div>
</div>
</div>
<div>Note the IDs: RegionOne has a46... and RegionTwo has e9c...</div>
<div><br>
</div>
<div>When I do "openstack endpoint show cinderv2", RegionTwo will be displayed. Why? Because "e" comes after "a". If I regenerate the endpoints until RegionOne has a UUID that is alphabetically later than RegionTwo, then RegionOne will be displayed.</div>
<div><br>
</div>
<div>It does not matter what my OS_REGION_NAME environment variable is set to, either.</div>
<div><br>
</div>
<div>Now, let's say my nova.conf file for the nova-api server in RegionOne does not have the</div>
<div><br>
</div>
<div>[cinder]</div>
<div>os_region_name = RegionOne</div>
<div><br>
</div>
<div>setting I mentioned earlier (also, note that this needs to go in a "[cinder]" section).</div>
<div><br>
</div>
<div>If RegionOne's UUID comes earlier in the alphabet (so RegionOne would NOT be displayed in "openstack endpoint show cinderv2"), then "nova volume-attach" works. nova-api will grab the correct endpoint for RegionOne.</div>
<div><br>
</div>
<div>If RegionOne's UUID comes later in the alphabet, then "nova volume-attach" will try to contact RegionTwo. </div>
<div><br>
</div>
<div>My guess is that the openstack client is reporting the last returned catalog entry while the nova-api service is using the first returned catalog entry.</div>
<div><br>
</div>
<div>BUT, if I set </div>
<div><br>
</div>
<div>
<div>[cinder]</div>
<div>os_region_name = RegionOne</div>
</div>
<div><br>
</div>
<div>then "nova volume-attach" works, no matter what order the entries are in the catalog.</div>
<div><br>
</div>
<div><br>
</div>
<div>Given all of the above, here is what is working for me:</div>
<div><br>
</div>
<div>1. I have two services in my Keystone catalog:</div>
<div><br>
</div>
<div>cinder: volume</div>
<div>cinderv2: volumev2</div>
<div><br>
</div>
<div>It looks like you have 4:</div>
<div><br>
</div>
<div>cinder: volume</div>
<div>cinderv2: volumev2</div>
<div><span style="font-size:13px">cinder-hpc2n: volume</span><br>
</div>
<div><span style="font-size:13px">cinderv2-hpc2n: volumev2</span><span style="font-size:13px"><br>
</span></div>
<div><span style="font-size:13px"><br>
</span></div>
<div><span style="font-size:13px">I don't know if that makes a difference, but maybe simplify the services to just the standard 2. </span></div>
<div><br>
</div>
<div>2. nova.conf on the nova-api server has the "os_region_name" setting set to the correct local region. I only had to restart the nova-api service to see changes take effect, but maybe restart nova-* -- who knows at this point!</div>
<div><br>
</div>
<div>3. Check the spelling of your regions. I noticed that you sometimes have "regionOne" and other times have "RegionOne" but maybe that is just a typo from sanitizing your data.</div>
<div><br>
</div>
<div>Let me know if this helps :)</div>
<div><br>
</div>
<div>Joe</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Thu, Dec 10, 2015 at 2:31 AM, Salman Toor <span dir="ltr">
<<a href="mailto:salman.toor@it.uu.se" target="_blank">salman.toor@it.uu.se</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">Hi Joe, 
<div><br>
</div>
<div>Thanks for your reply, yes Me and Marco are working together. </div>
<div><br>
</div>
<div>I have tried to used </div>
<div><br>
</div>
<div>os_region_name = RegionOne</div>
<div><br>
</div>
<div>both in cinder.conf and nova.conf but no luck so far. But the error message is a bit different. It seems like now the system is recognising the region. </div>
<div><br>
</div>
<div><br>
</div>
<div>
<p style="font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
<font color="#e32400">2015-12-10 09:32:54.661 26792 TRACE nova.api.openstack EndpointNotFound: internalURL endpoint for volume service named cinder in regionOne region not found</font></p>
<div><br>
</div>
</div>
<div>Following are my endpoints:</div>
<div><br>
</div>
<div>——</div>
<div>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
<b>[root@controller: ~]</b> # openstack endpoint show cinder</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
+--------------+------------------------------------------------+</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| Field        | Value                                          |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
+--------------+------------------------------------------------+</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| adminurl     | <a href="http://cinder:8776/v1/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">http://cinder:8776/v1/%(tenant_id)s</a>            |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| enabled      | True                                           |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| id           | e2693bcaf3da4be2810f04acd7995d7f               |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| internalurl  | <a href="http://cinder:8776/v1/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">http://cinder:8776/v1/%(tenant_id)s</a>            |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| publicurl    | <a href="http://smog.uppmax.uu.se:8776/v1/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">http://</a><a href="http://cinder:8776/v1/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">cinder</a><a href="http://smog.uppmax.uu.se:8776/v1/%(tenant_id)s" style="font-size:12.8px;color:rgb(17,85,204)" target="_blank">:8776/v1/%(tenant_id)s</a><span style="font-size:12.8px"> </span><span style="font-size:12.8px">|</span></p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| region       | regionOne                                      |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| service_id   | 7bd5d667ec4b4d65b6c1b0de8b303fe3               |</p>
<span>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| service_name | cinder                                         |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| service_type | volume                                         |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
+--------------+------------------------------------------------+</p>
</span>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
<b>[root@controller: ~]</b> # openstack endpoint show cinderv2</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
+--------------+------------------------------------------------+</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| Field        | Value                                          |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
+--------------+------------------------------------------------+</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| adminurl     | <a href="http://cinder:8776/v2/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">http://cinder:8776/v2/%(tenant_id)s</a>            |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| enabled      | True                                           |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| id           | 3000cb23c6ab4ee5b68876ee08257338               |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| internalurl  | <a href="http://cinder:8776/v2/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">http://cinder:8776/v2/%(tenant_id)s</a>            |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| publicurl    | <a href="http://smog.uppmax.uu.se:8776/v2/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">http://</a><a href="http://cinder:8776/v1/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">cinder</a><a href="http://smog.uppmax.uu.se:8776/v2/%(tenant_id)s" style="font-size:12.8px;color:rgb(17,85,204)" target="_blank">:8776/v2/%(tenant_id)s</a><span style="font-size:12.8px"> </span><span style="font-size:12.8px">|</span></p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| region       | regionOne                                      |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| service_id   | 61d8baeb4ee74c7798a60758b2f4171f               |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| service_name | cinderv2                                       |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| service_type | volumev2                                       |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
+--------------+------------------------------------------------+</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
<b>[root@controller: ~]</b> # openstack endpoint show cinder-hpc2n</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
+--------------+--------------------------------------------------+</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| Field        | Value                                            |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
+--------------+--------------------------------------------------+</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| adminurl     | <a href="http://u-mn01.hpc2n.umu.se:8776/v1/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">http://</a><a href="http://cinder:8776/v1/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">cinder</a><a href="http://u-mn01.hpc2n.umu.se:8776/v1/%(tenant_id)s" style="font-size:12.8px;color:rgb(17,85,204)" target="_blank">:8776/v1/%(tenant_id)s</a><span style="font-size:12.8px"> </span><span style="font-size:12.8px">|</span></p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| enabled      | True                                             |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| id           | c088f1f9ca0248a08eb60d4ac8492150                 |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| internalurl  | <a href="http://u-mn01.hpc2n.umu.se:8776/v1/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">http://</a><a href="http://cinder:8776/v1/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">cinder</a><a href="http://u-mn01.hpc2n.umu.se:8776/v1/%(tenant_id)s" style="font-size:12.8px;color:rgb(17,85,204)" target="_blank">:8776/v1/%(tenant_id)s</a><span style="font-size:12.8px"> </span><span style="font-size:12.8px">|</span></p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| publicurl    | <a href="http://u-mn01.hpc2n.umu.se:8776/v1/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">http://</a><a href="http://cinder:8776/v1/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">cinder</a><a href="http://u-mn01.hpc2n.umu.se:8776/v1/%(tenant_id)s" style="font-size:12.8px;color:rgb(17,85,204)" target="_blank">:8776/v1/%(tenant_id)s</a><span style="font-size:12.8px"> </span><span style="font-size:12.8px">|</span></p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| region       | HPC2N                                            |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| service_id   | 473f3eab043f4130b30d9ee64fe43703                 |</p>
<span>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| service_name | cinder-hpc2n                                     |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| service_type | volume                                           |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
+--------------+--------------------------------------------------+</p>
</span>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
<b>[root@controller: ~]</b> # openstack endpoint show cinderv2-hpc2n</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
+--------------+--------------------------------------------------+</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| Field        | Value                                            |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
+--------------+--------------------------------------------------+</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| adminurl     | <a href="http://u-mn01.hpc2n.umu.se:8776/v2/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">http://</a><a href="http://cinder:8776/v1/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">cinder</a><a href="http://u-mn01.hpc2n.umu.se:8776/v2/%(tenant_id)s" style="font-size:12.8px;color:rgb(17,85,204)" target="_blank">:8776/v2/%(tenant_id)s</a><span style="font-size:12.8px"> </span><span style="font-size:12.8px">|</span></p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| enabled      | True                                             |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| id           | 985e5b8b068e4c3d99c12788236ee0fe                 |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| internalurl  | <a href="http://u-mn01.hpc2n.umu.se:8776/v2/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">http://</a><a href="http://cinder:8776/v1/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">cinder</a><a href="http://u-mn01.hpc2n.umu.se:8776/v2/%(tenant_id)s" style="font-size:12.8px;color:rgb(17,85,204)" target="_blank">:8776/v2/%(tenant_id)s</a><span style="font-size:12.8px"> </span><span style="font-size:12.8px">|</span></p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| publicurl    | <a href="http://u-mn01.hpc2n.umu.se:8776/v2/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">http://</a><a href="http://cinder:8776/v1/%(tenant_id)s" style="color:rgb(17,85,204)" target="_blank">cinder</a><a href="http://u-mn01.hpc2n.umu.se:8776/v2/%(tenant_id)s" style="font-size:12.8px;color:rgb(17,85,204)" target="_blank">:8776/v2/%(tenant_id)s</a><span style="font-size:12.8px"> </span><span style="font-size:12.8px">|</span></p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| region       | HPC2N                                            |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| service_id   | 089186fad4d7444b95e74ea9d7769423                 |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| service_name | cinderv2-hpc2n                                   |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
| service_type | volumev2                                         |</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
<br>
</p>
<p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;background-color:rgb(255,255,255)">
+--------------+--------------------------------------------------+</p>
</div>
<div>——</div>
<div><br>
</div>
<div>Again if I remove hpc2n everything started to work fine. Can you tell me what service I suppose to restart (if required)?</div>
<div><br>
</div>
<div>Regards..</div>
<span><font color="#888888">
<div>Salman</div>
<div><br>
</div>
<div><br>
</div>
</font></span>
<div><span><br>
<div>
<div style="letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word">
PhD, Scientific Computing<br>
Researcher, IT Department,<br>
Uppsala University.<br>
Senior Cloud Architect, <br>
SNIC.<br>
Cloud Application Expert,<br>
UPPMAX.<br>
Visiting Researcher,<br>
Helsinki Institute of Physics (HIP).<br>
<a href="mailto:salman.toor@it.uu.se" target="_blank">salman.toor@it.uu.se</a><br>
<a href="http://www.it.uu.se/katalog/salto690" target="_blank">http://www.it.uu.se/katalog/salto690</a></div>
</div>
<br>
</span>
<div>
<div>
<div>
<blockquote type="cite">
<div>On 09 Dec 2015, at 17:48, Joe Topjian <<a href="mailto:joe@topjian.net" target="_blank">joe@topjian.net</a>> wrote:</div>
<br>
<div>
<div dir="ltr">Hi Salman,
<div><br>
</div>
<div>Someone mentioned this same issue yesterday in relation to Terraform (maybe a colleague of yours?), so given the two occurrences, I thought I'd look into this.</div>
<div><br>
</div>
<div>I have a Liberty environment readily available, so I created a second set of volume and volumev2 endpoints for a fictional region. Everything worked as expected, so I started reviewing the config files and saw that /etc/cinder/cinder.conf had an option</div>
<div><br>
</div>
<div>[DEFAULT]</div>
<div>os_region_name = RegionOne</div>
<div><br>
</div>
<div>I commented that out, but things still worked.</div>
<div><br>
</div>
<div>Then in /etc/nova/nova.conf, I saw:</div>
<div><br>
</div>
<div>[cinder]</div>
<div>os_region_name = RegionOne</div>
<div><br>
</div>
<div>commenting this out caused volume attachments to hang indefinitely because nova was trying to contact cinder at RegionTwo (I'm assuming this is the first catalog entry that was returned).</div>
<div><br>
</div>
<div>Given this is a Liberty environment, it's not accurately reproducing your problem, but could you check and see if you have that option set in nova.conf?</div>
<div><br>
</div>
<div>I have a Kilo environment in the process of building. Once it has finished, I'll see if I can reproduce your error there.</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Joe</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Wed, Dec 9, 2015 at 4:35 AM, Salman Toor <span dir="ltr">
<<a href="mailto:salman.toor@it.uu.se" target="_blank">salman.toor@it.uu.se</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">
<div>Hi, </div>
<div><br>
</div>
<div>I am using Kilo release on CentOS. We have recently enabled multiple regions and it seems that Cinder have some problems with  multiple endpoints. </div>
<div><br>
</div>
<div>Thinks are working fine with nova but cinder is behaving strange. Here are my endpoints </div>
<div><br>
</div>
<div><br>
</div>
<div>————</div>
<div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">
<div style="margin:0px;line-height:normal"><span style="color:#5330e1"><b>[root@controller: ~]</b></span> # openstack service list</div>
<div style="margin:0px;line-height:normal">+----------------------------------+----------------+--------------+</div>
<div style="margin:0px;line-height:normal">| ID                               | Name           | Type         |</div>
<div style="margin:0px;line-height:normal">+----------------------------------+----------------+--------------+</div>
<div style="margin:0px;line-height:normal"><font color="#e32400">| 0a33e6f259794ff2a99e626be37c0c2b | cinderv2-hpc2n | volumev2     |</font></div>
<div style="margin:0px;line-height:normal">| 1fcae9bd76304853a3168c39c7fe8e6b | nova           | compute      |</div>
<div style="margin:0px;line-height:normal">| 2c7828120c294d3f82e3a17835babb85 | neutron        | network      |</div>
<div style="margin:0px;line-height:normal">| 3804fcd8f9494d30b589b55fe6abb811 | nova-hpc2n     | compute      |</div>
<div style="margin:0px;line-height:normal">| 478eff4e96464ae8a958ba29f750b14c | glance         | image        |</div>
<div style="margin:0px;line-height:normal"><font color="#e32400">| 4a5a771d915e43c28e66538b8bc6e625 | cinder         | volume       |</font></div>
<div style="margin:0px;line-height:normal"><font color="#e32400">| 72d1be82b2e5478dbf0f3fb9e7ba969d | cinderv2       | volumev2     |</font></div>
<div style="margin:0px;line-height:normal">| 97f977f8a7a04bae89da167fd25dc06c | glance-hpc2n   | image        |</div>
<div style="margin:0px;line-height:normal"><font color="#e32400">| a985795b49e2440db82970b81248c86e | cinder-hpc2n   | volume       |</font></div>
<div style="margin:0px;line-height:normal">| dccd39b92ab547ddaf9047b38620145a | swift          | object-store |</div>
<div style="margin:0px;line-height:normal">| ebb1660d1d9746759a48de921521bfad | keystone       | identity     |</div>
<div style="margin:0px;line-height:normal">+----------------------------------+----------------+--------------+</div>
</div>
</div>
<div><br>
</div>
<div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><span style="color:#5330e1"><b>[root@controller: ~]</b></span> # openstack endpoint show a985795b49e2440db82970b81248c86e</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">+--------------+--------------------------------------------------+</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| Field        | Value                                            |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">+--------------+--------------------------------------------------+</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| adminurl     | http://<URL>:8776/v1/%(tenant_id)s |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| enabled      | True                                             |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| id           | d4003e91ddf24cfb9fa497da81b01a18                 |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| internalurl  | http://<URL>:8776/v1/%(tenant_id)s |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| publicurl    | http://<URL>:8776/v1/%(tenant_id)s |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| region       | HPC2N                                            |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| service_id   | a985795b49e2440db82970b81248c86e                 |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| service_name | cinder-hpc2n                                     |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| service_type | volume                                           |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">+--------------+--------------------------------------------------+</div>
</div>
<div><br>
</div>
<div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><span style="color:#5330e1"><b>[root@controller: ~]</b></span> # openstack endpoint show 4a5a771d915e43c28e66538b8bc6e625</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">+--------------+------------------------------------------------+</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| Field        | Value                                          |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">+--------------+------------------------------------------------+</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| adminurl     | http://<URL>:8776/v1/%(tenant_id)s            |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| enabled      | True                                           |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| id           | 5f19c0b535674dbd9e318c7b6d61b3bc               |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| internalurl  | http://<URL>:8776/v1/%(tenant_id)s            |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| publicurl    | http://<URL>:8776/v1/%(tenant_id)s |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| region       | regionOne                                      |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| service_id   | 4a5a771d915e43c28e66538b8bc6e625               |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| service_name | cinder                                         |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">| service_type | volume                                         |</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">+--------------+------------------------------------------------+</div>
</div>
<div><br>
</div>
<div>And same for v2 endpoints  </div>
<div><br>
</div>
<div>————</div>
<div><br>
</div>
<div>——— nova-api.log ———</div>
<div><br>
</div>
<div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">achmentController object at 0x598a3d0>>, body: {"volumeAttachment": {"device": "", "volumeId": "93d96eab-e3fd-4131-9549-ed51e7299da2"}} _process_stack /usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py:780</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.847 3376 INFO nova.api.openstack.compute.contrib.volumes [req-d6e1380d-c6bc-4911-b2c6-251bc8b4c62c a62c20fdf99c443a924f0d50a51514b1 3c9d997982e04c6db0e02b82fa18fdd8
 - - -] Attach volume 93d96eab-e3fd-4131-9549-ed51e7299da2 to instance 3a4c8722-52a7-48f2-beb7-db8938698a0d at </div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.847 3376 DEBUG nova.compute.api [req-d6e1380d-c6bc-4911-b2c6-251bc8b4c62c a62c20fdf99c443a924f0d50a51514b1 3c9d997982e04c6db0e02b82fa18fdd8 - - -] [instance:
 3a4c8722-52a7-48f2-beb7-db8938698a0d] Fetching instance by UUID get /usr/lib/python2.7/site-packages/nova/compute/api.py:1911</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 ERROR nova.api.openstack [req-d6e1380d-c6bc-4911-b2c6-251bc8b4c62c a62c20fdf99c443a924f0d50a51514b1 3c9d997982e04c6db0e02b82fa18fdd8 - - -] Caught error:
 internalURL endpoint for volume service named cinder not found</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack Traceback (most recent call last):</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/api/openstack/__init__.py", line 125, in __call__</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     return req.get_response(self.application)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1296, in send</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     application, catch_exc_info=False)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1260, in call_application</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     app_iter = application(self.environ, start_response)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 144, in __call__</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     return resp(environ, start_response)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 634, in __call__</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     return self._call_app(env, start_response)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 554, in _call_app</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     return self._app(env, _fake_start_response)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 144, in __call__</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     return resp(environ, start_response)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 144, in __call__</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     return resp(environ, start_response)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/routes/middleware.py", line 131, in __call__</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     response = self.app(environ, start_response)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 144, in __call__</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     return resp(environ, start_response)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 130, in __call__</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     resp = self.call_func(req, *args, **self.kwargs)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 195, in call_func</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     return self.func(req, *args, **kwargs)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py", line 756, in __call__</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     content_type, body, accept)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py", line 821, in _process_stack</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     action_result = self.dispatch(meth, request, action_args)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py", line 911, in dispatch</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     return method(req=request, **action_args)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/contrib/volumes.py", line 305, in create</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     volume_id, device)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 230, in wrapped</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     return func(self, context, target, *args, **kwargs)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 219, in inner</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     return function(self, context, instance, *args, **kwargs)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 200, in inner</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     return f(self, context, instance, *args, **kw)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 3035, in attach_volume</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     disk_bus, device_type)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 3016, in _attach_volume</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     volume_bdm.destroy()</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     six.reraise(self.type_, self.value, self.tb)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 3009, in _attach_volume</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     volume = self.volume_api.get(context, volume_id)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/volume/cinder.py", line 214, in wrapper</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     res = method(self, ctx, volume_id, *args, **kwargs)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/volume/cinder.py", line 286, in get</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     item = cinderclient(context).volumes.get(volume_id)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/volume/cinder.py", line 115, in cinderclient</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     url = _SESSION.get_endpoint(auth, **service_parameters)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 635, in get_endpoint</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     return auth.get_endpoint(self, **kwargs)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystoneclient/auth/identity/base.py", line 219, in get_endpoint</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     service_name=service_name)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystoneclient/utils.py", line 318, in inner</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     return func(*args, **kwargs)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystoneclient/service_catalog.py", line 254, in url_for</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack     raise exceptions.EndpointNotFound(msg)</div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo"><font color="#ff4013">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack EndpointNotFound: internalURL endpoint for volume service named cinder not found</font></div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">2015-12-08 12:52:05.988 3376 TRACE nova.api.openstack</div>
</div>
<div><br>
</div>
<div>———</div>
<div><br>
</div>
<div>If I remove cinder-hpc2n and cinderv2-hpc2n from the service list, everything started to work fine. </div>
<div><br>
</div>
<div>This is how I have created endpoints: </div>
<div><br>
</div>
<div><br>
</div>
<div>
<div style="margin:0px;font-size:11px;line-height:normal;font-family:Menlo">openstack endpoint create --publicurl http://<URL>:8776/v1/%\(tenant_id\)s --internalurl http://<URL>:8776/v1/%\(tenant_id\)s --adminurl http://<URL>:8776/v1/%\(tenant_id\)s --region
 regionOne 4a5a771d915e43c28e66538b8bc6e625 </div>
</div>
<div><br>
</div>
<div><span style="font-family:Menlo;font-size:11px">openstack endpoint create --publicurl http://<URL>:8776/v1/%\(tenant_id\)s --internalurl http://<URL>:8776/v1/%\(tenant_id\)s --adminurl http://<URL>:8776/v1/%\(tenant_id\)s --region HPC2N </span><span style="font-family:Menlo;font-size:11px">a985795b49e2440db82970b81248c86e</span></div>
<div><br>
</div>
<div>using the service-id. </div>
<div><br>
</div>
<div><br>
</div>
<div>Anyone experiencing this behavior? or Any suggestion how to fix it? Again regions are working fine with nova and glance. </div>
<div><br>
</div>
<div><br>
</div>
<div>Regards..</div>
<div>Salman. </div>
<div><br>
</div>
<div><br>
</div>
<div>
<div style="letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word">
<div>PhD, Scientific Computing<br>
Researcher, IT Department,<br>
Uppsala University.</div>
<div>
<div style="word-wrap:break-word">Senior Cloud Architect, </div>
<div style="word-wrap:break-word">SNIC.</div>
Cloud Application Expert,<br>
UPPMAX.<br>
Visiting Researcher,<br>
Helsinki Institute of Physics (HIP).<br>
<a href="mailto:salman.toor@it.uu.se" target="_blank">salman.toor@it.uu.se</a><br>
<a href="http://www.it.uu.se/katalog/salto690" target="_blank">http://www.it.uu.se/katalog/salto690</a></div>
</div>
</div>
<br>
</div>
<br>
_______________________________________________<br>
OpenStack-operators mailing list<br>
<a href="mailto:OpenStack-operators@lists.openstack.org" target="_blank">OpenStack-operators@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div></div></div>
</div>
</div>
</div>

</blockquote></div><br></div>