<div dir="ltr"><div>Gilles, <br><br>I already considered this when looking at another openstackclient issue. Version 1.0.4 has almost no changes from 1.0.3, which is the official release for Kilo. Maybe we can get this keystone URL handling fix backported to the 1.0.X branch of openstackclient?<br><br></div>-Matthew<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Aug 14, 2015 at 3:47 PM, Gilles Dubreuil <span dir="ltr"><<a href="mailto:gilles@redhat.com" target="_blank">gilles@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5"><br>
<br>
On 14/08/15 20:45, Gilles Dubreuil wrote:<br>
><br>
><br>
> On 13/08/15 23:29, Rich Megginson wrote:<br>
>> On 08/13/2015 12:41 AM, Gilles Dubreuil wrote:<br>
>>> Hi Matthew,<br>
>>><br>
>>> On 11/08/15 01:14, Rich Megginson wrote:<br>
>>>> On 08/10/2015 07:46 AM, Matthew Mosesohn wrote:<br>
>>>>> Sorry to everyone for bringing up this old thread, but it seems we may<br>
>>>>> need more openstackclient/keystone experts to settle this.<br>
>>>>><br>
>>>>> I'm referring to the comments in<br>
>>>>> <a href="https://review.openstack.org/#/c/207873/" rel="noreferrer" target="_blank">https://review.openstack.org/#/c/207873/</a><br>
>>>>> Specifically comments from Richard Megginson and Gilles Dubreuil<br>
>>>>> indicating openstackclient behavior for v3 keystone API.<br>
>>>>><br>
>>>>><br>
>>>>> A few items seem to be under dispute:<br>
>>>>> 1 - Keystone should be able to accept v3 requests at<br>
>>>>> <<a href="http://keystone-server:5000/" rel="noreferrer" target="_blank">http://keystone-server:5000/</a>><a href="http://keystone-server:5000/" rel="noreferrer" target="_blank">http://keystone-server:5000/</a><br>
>>>> I don't think so. Keystone requires the version suffix "/v2.0" or<br>
>>>> "/v3".<br>
>>>><br>
>>> Yes, if the public endpoint is set without a version then the service<br>
>>> can deal with either version.<br>
>>><br>
>>> <a href="http://paste.openstack.org/raw/412819/" rel="noreferrer" target="_blank">http://paste.openstack.org/raw/412819/</a><br>
>>><br>
>>> That is not true for the admin endpoint (authentication is already done,<br>
>>> the admin services deals only with tokens), at least for now, Keystone<br>
>>> devs are working on it.<br>
>><br>
>> I thought it worked like this - the openstack cli will infer from the<br>
>> arguments if it should do v2 or v3 auth. In the above case for v3,<br>
>> since you specify the username/password, osc knows it has to use<br>
>> password auth (as opposed to token auth), and since all of the required<br>
>> v3 arguments are provided (v3 api version, domains for user/project), it<br>
>> can use v3 password auth. It knows it has to use the "/v3/auth/token"<br>
>> path to get a token.<br>
>><br>
>> Similarly for v2, since it only has username/password, no v3 api or v3<br>
>> domain arguments, it knows it has to use v2 password auth. It knows it<br>
>> has to use "/v2.0/token" to get a token.<br>
>><br>
>> With the puppet-keystone code, since it uses TOKEN/URL, osc cannot infer<br>
>> if it can use v2 or v3, so it requires the "/v2.0" or "/v3" suffix, and<br>
>> the api version.<br>
>><br>
><br>
> First of my apologies because I confused admin enpdoint with the admin<br>
> service (or whatever that's dubbed).<br>
><br>
> As of Keystone v3 (not the API, the latest version of Keystone which<br>
> supports both API v2.0 and V3), the OS_AUTH_URL doesn't need the<br>
> version. That can be effectively any of the endpoints, either the main<br>
> (or public) by default on port 5000 or the admin by default on port<br>
> 35357, the third one internal pointing to either of the first two ones.<br>
><br>
> This is a behavior at Keystone level not at the OSC. I mean OSC will<br>
> just reflect the http-api behavior.<br>
><br>
> Now the admin service, the one needed for the OS-TOKEN (used for<br>
> bootstrapping) needs a URL (OS_URL) with a version to work.<br>
><br>
> ATM, the issue with puppet keystone is that endpoints, OS_URL and<br>
> OS_AUTH_URL are walking on each others.<br>
><br>
><br>
<br>
</div></div>My latest update on this one, is that I found out while testing beaker<br>
which is using OSC 1.0.3 is not handling OS_AUTH_URL properly.<br>
<br>
I had been testing with OSC 1.5.1 and now latest 1.6.1 from Dolerean<br>
repo, where the version-less URLs are working, but not with OSC 1.0.1:<br>
<br>
----------------------<br>
<br>
# cat openrc<br>
export OS_AUTH_URL=<a href="http://127.0.0.1:5000" rel="noreferrer" target="_blank">http://127.0.0.1:5000</a><br>
export OS_USERNAME=adminv3<br>
export OS_PASSWORD=testing<br>
export OS_PROJECT_NAME=openstackv3<br>
export OS_USER_DOMAIN_NAME=admin_domain<br>
export OS_PROJECT_DOMAIN_NAME=admin_domain<br>
export OS_IDENTITY_API_VERSION="3"<br>
<br>
# openstack endpoint list -f csv<br>
"ID","Region","Service Name","Service Type","Enabled","Interface","URL"<br>
"87b7db1b23df487bb4ec96de5aa3c271","RegionOne","keystone","identity",True,"internal","<a href="http://127.0.0.1:5000" rel="noreferrer" target="_blank">http://127.0.0.1:5000</a>"<br>
"d9b345109d8a4320ac0dd832d2532cce","RegionOne","keystone","identity",True,"admin","<a href="http://127.0.0.1:35357" rel="noreferrer" target="_blank">http://127.0.0.1:35357</a>"<br>
"f3a579a64f0241ef9aef3dc983e0fd4a","RegionOne","keystone","identity",True,"public","<a href="http://127.0.0.1:5000" rel="noreferrer" target="_blank">http://127.0.0.1:5000</a>"<br>
<br>
----------------------<br>
<br>
# cat openrc_v2<br>
export OS_AUTH_URL=http://[::1]:5000<br>
export OS_USERNAME=admin<br>
export OS_PASSWORD=a_big_secret<br>
export OS_TENANT_NAME=openstack<br>
<br>
# openstack endpoint list -f csv --long<br>
"ID","Region","Service Name","Service<br>
Type","PublicURL","AdminURL","InternalURL"<br>
"cf8825c44bd041109d5c7438ba9db8f3","RegionOne","keystone","identity","<a href="http://127.0.0.1:5000" rel="noreferrer" target="_blank">http://127.0.0.1:5000</a>","<a href="http://127.0.0.1:35357" rel="noreferrer" target="_blank">http://127.0.0.1:35357</a>","<a href="http://127.0.0.1:5000" rel="noreferrer" target="_blank">http://127.0.0.1:5000</a>"<br>
<div class="HOEnZb"><div class="h5"><br>
<br>
<br>
<br>
>>><br>
>>>>> 2 - openstackclient should be able to interpret v3 requests and append<br>
>>>>> "v3/" to OS_AUTH_URL=<a href="http://keystone-server.5000/" rel="noreferrer" target="_blank">http://keystone-server.5000/</a> or rewrite the URL<br>
>>>>> if it is set as<br>
>>>>> OS_AUTH_URL=<<a href="http://keystone-server.5000/" rel="noreferrer" target="_blank">http://keystone-server.5000/</a>><a href="http://keystone-server.5000/" rel="noreferrer" target="_blank">http://keystone-server.5000/</a><br>
>>>> It does, if it can determine from the given authentication arguments if<br>
>>>> it can do v3 or v2.0.<br>
>>>><br>
>>> It effectively does, again, assuming the path doesn't contain a version<br>
>>> number (x.x.x.x:5000)<br>
>>><br>
>>>>> 3 - All deployments require /etc/keystone/keystone.conf with a token<br>
>>>>> (and not simply use openrc for creating additional endpoints, users,<br>
>>>>> etc beyond keystone itself and an admin user)<br>
>>>> No. What I said about this issue was "Most people using<br>
>>>> puppet-keystone, and realizing Keystone resources on nodes that are not<br>
>>>> the Keystone node, put a /etc/keystone/keystone.conf on that node with<br>
>>>> the admin_token in it."<br>
>>>><br>
>>>> That doesn't mean the deployment requires /etc/keystone/keystone.conf.<br>
>>>> It should be possible to realize Keystone resources on non-Keystone<br>
>>>> nodes by using ENV or openrc or other means.<br>
>>>><br>
>>> Agreed. Also keystone.conf is used only to bootstrap keystone<br>
>>> installation and create admin users, etc.<br>
>>><br>
>>><br>
>>>>> I believe it should be possible to set v2.0 keystone OS_AUTH_URL in<br>
>>>>> openrc and puppet-keystone + puppet-openstacklib should be able to<br>
>>>>> make v3 requests sensibly by manipulating the URL.<br>
>>>> Yes. Because for the puppet-keystone resource providers, they are coded<br>
>>>> to a specific version of the api, and therefore need to be able to<br>
>>>> set/override the OS_IDENTITY_API_VERSION and the version suffix in<br>
>>>> the URL.<br>
>>>><br>
>>> No. To support V2 and V3, the OS_AUTH_URL must not contain any version<br>
>>> in order.<br>
>>><br>
>>> The less we deal with the version number the better!<br>
>>><br>
>>>>> Additionally, creating endpoints/users/roles shouldbe possible via<br>
>>>>> openrc alone.<br>
>>>> Yes.<br>
>>>><br>
>>> Yes, the openrc variables are used, if not available then the service<br>
>>> token from the keystone.conf is used.<br>
>>><br>
>>>>> It's not possible to write single node tests that can demonstrate this<br>
>>>>> functionality, which is why it probably went undetected for so long.<br>
>>>> And since this is supported, we need tests for this.<br>
>>> I'm not sure what's the issue besides the fact keystone_puppet doesn't<br>
>>> generate a RC file once the admin user has been created. That is left to<br>
>>> be done by the composition layer. Although we might want to integrate<br>
>>> that.<br>
>>><br>
>>> If that issue persists, assuming the AUTH_URL is free for a version<br>
>>> number and having an openrc in place, we're going to need a bug number<br>
>>> to track the investigation.<br>
>>><br>
>>>>> If anyone can speak up on these items, it could help influence the<br>
>>>>> outcome of this patch.<br>
>>>>><br>
>>>>> Thank you for your time.<br>
>>>>><br>
>>>>> Best Regards,<br>
>>>>> Matthew Mosesohn<br>
>>><br>
>>> Thanks,<br>
>>> Gilles<br>
>>><br>
>>>>> On Fri, Jul 31, 2015 at 6:32 PM, Rich Megginson <<a href="mailto:rmeggins@redhat.com">rmeggins@redhat.com</a><br>
>>>>> <mailto:<a href="mailto:rmeggins@redhat.com">rmeggins@redhat.com</a>>> wrote:<br>
>>>>><br>
>>>>> On 07/31/2015 07:18 AM, Matthew Mosesohn wrote:<br>
>>>>><br>
>>>>> Jesse, thanks for raising this. Like you, I should just track<br>
>>>>> upstream<br>
>>>>> and wait for full V3 support.<br>
>>>>><br>
>>>>> I've taken the quickest approach and written fixes to<br>
>>>>> puppet-openstacklib and puppet-keystone:<br>
>>>>> <a href="https://review.openstack.org/#/c/207873/" rel="noreferrer" target="_blank">https://review.openstack.org/#/c/207873/</a><br>
>>>>> <a href="https://review.openstack.org/#/c/207890/" rel="noreferrer" target="_blank">https://review.openstack.org/#/c/207890/</a><br>
>>>>><br>
>>>>> and again to Fuel-Library:<br>
>>>>> <a href="https://review.openstack.org/#/c/207548/1" rel="noreferrer" target="_blank">https://review.openstack.org/#/c/207548/1</a><br>
>>>>><br>
>>>>> I greatly appreciate the quick support from the community to<br>
>>>>> find an<br>
>>>>> appropriate solution. Looks like I'm just using a weird<br>
>>>>> edge case<br>
>>>>> where we're creating users on a separate node from where<br>
>>>>> keystone is<br>
>>>>> installed and it never got thoroughly tested, but I'm happy<br>
>>>>> to fix<br>
>>>>> bugs where I can.<br>
>>>>><br>
>>>>><br>
>>>>> Most puppet deployments either realize all keystone resources on<br>
>>>>> the keystone node, or drop an /etc/keystone/keystone.conf with<br>
>>>>> admin token onto non-keystone nodes where additional keystone<br>
>>>>> resources need to be realized.<br>
>>>>><br>
>>>>><br>
>>>>><br>
>>>>> -Matthew<br>
>>>>><br>
>>>>> On Fri, Jul 31, 2015 at 3:54 PM, Jesse Pretorius<br>
>>>>> <<a href="mailto:jesse.pretorius@gmail.com">jesse.pretorius@gmail.com</a> <mailto:<a href="mailto:jesse.pretorius@gmail.com">jesse.pretorius@gmail.com</a>>><br>
>>>>> wrote:<br>
>>>>><br>
>>>>> With regards to converting all services to use Keystone v3<br>
>>>>> endpoints, note<br>
>>>>> the following:<br>
>>>>><br>
>>>>> 1) swift-dispersion currently does not support consuming<br>
>>>>> Keystone v3<br>
>>>>> endpoints [1]. There is a patch merged to master [2] to<br>
>>>>> fix that, but a<br>
>>>>> backport to kilo is yet to be done.<br>
>>>>> 2) Each type (internal, admin, public) of endpoint created<br>
>>>>> with the Keystone<br>
>>>>> v3 API has its own unique id, unlike with the v2 API where<br>
>>>>> they're all<br>
>>>>> created with a single ID. This results in the keystone<br>
>>>>> client being unable<br>
>>>>> to read the catalog created via the v3 API when querying<br>
>>>>> via the v2 API. The<br>
>>>>> solution is to use the openstack client and to use the v3<br>
>>>>> API but this<br>
>>>>> obviously needs to be noted for upgrade impact and<br>
>>>>> operators.<br>
>>>>> 3) When glance is setup to use swift as a back-end,<br>
>>>>> glance_store is unable<br>
>>>>> to authenticate to swift when the endpoint it uses is a v3<br>
>>>>> endpoint. There<br>
>>>>> is a review to master in progress [3] to fix this which is<br>
>>>>> unlikely to make<br>
>>>>> it into kilo.<br>
>>>>><br>
>>>>> We (the openstack-ansible/os-ansible-deployment project)<br>
>>>>> are tracking these<br>
>>>>> issues and doing tests to figure out all the bits. These<br>
>>>>> are the bugs we've<br>
>>>>> hit so far. Also note that there is a WIP patch to gate<br>
>>>>> purely on Keystone<br>
>>>>> v3 API's which is planned to become voting (hopefully) by<br>
>>>>> the end of this<br>
>>>>> cycle.<br>
>>>>><br>
>>>>> [1] <a href="https://bugs.launchpad.net/swift/+bug/1468374" rel="noreferrer" target="_blank">https://bugs.launchpad.net/swift/+bug/1468374</a><br>
>>>>> [2] <a href="https://review.openstack.org/195131" rel="noreferrer" target="_blank">https://review.openstack.org/195131</a><br>
>>>>> [3] <a href="https://review.openstack.org/193422" rel="noreferrer" target="_blank">https://review.openstack.org/193422</a><br>
>>>>><br>
>>>>><br>
>>>>> __________________________________________________________________________<br>
>>>>><br>
>>>>> OpenStack Development Mailing List (not for usage<br>
>>>>> questions)<br>
>>>>> Unsubscribe:<br>
>>>>><br>
>>>>> <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
>>>>><br>
>>>>> <<a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>><br>
>>>>><br>
>>>>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
>>>>><br>
>>>>><br>
>>>>> __________________________________________________________________________<br>
>>>>><br>
>>>>> OpenStack Development Mailing List (not for usage questions)<br>
>>>>> Unsubscribe:<br>
>>>>> <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
>>>>><br>
>>>>> <<a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>><br>
>>>>><br>
>>>>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
>>>>><br>
>>>>><br>
>>>>><br>
>>>>><br>
>>>>> __________________________________________________________________________<br>
>>>>><br>
>>>>> OpenStack Development Mailing List (not for usage questions)<br>
>>>>> Unsubscribe:<br>
>>>>> <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
>>>>><br>
>>>>> <<a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>><br>
>>>>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
>>>>><br>
>>>>><br>
>>>>><br>
>>>>><br>
>>>>> __________________________________________________________________________<br>
>>>>><br>
>>>>> OpenStack Development Mailing List (not for usage questions)<br>
>>>>> Unsubscribe:<br>
>>>>> <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
>>>>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
>>>><br>
>>>><br>
>>>> __________________________________________________________________________<br>
>>>><br>
>>>> OpenStack Development Mailing List (not for usage questions)<br>
>>>> Unsubscribe:<br>
>>>> <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
>>>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
>>>><br>
>>> __________________________________________________________________________<br>
>>><br>
>>> OpenStack Development Mailing List (not for usage questions)<br>
>>> Unsubscribe:<br>
>>> <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
>>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
>><br>
>><br>
>> __________________________________________________________________________<br>
>> OpenStack Development Mailing List (not for usage questions)<br>
>> Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
><br>
> __________________________________________________________________________<br>
> OpenStack Development Mailing List (not for usage questions)<br>
> Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
><br>
<br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</div></div></blockquote></div><br></div>