[openstack-dev] [Fuel][Puppet] Keystone V2/V3 service endpoints
Rich Megginson
rmeggins at redhat.com
Fri Aug 14 14:23:09 UTC 2015
On 08/14/2015 06:51 AM, Matthew Mosesohn wrote:
> Gilles,
>
> I already considered this when looking at another openstackclient
> issue. Version 1.0.4 has almost no changes from 1.0.3, which is the
> official release for Kilo. Maybe we can get this keystone URL handling
> fix backported to the 1.0.X branch of openstackclient?
I think we need some sort of fix in openstacklib and/or puppet-keystone
so that v3 providers that use token auth can replace any "/v2.0" in the
url with "/v3", to override any settings of OS_URL or OS_AUTH_URL in ENV
or openrc.
>
> -Matthew
>
> On Fri, Aug 14, 2015 at 3:47 PM, Gilles Dubreuil <gilles at redhat.com
> <mailto:gilles at redhat.com>> wrote:
>
>
>
> On 14/08/15 20:45, Gilles Dubreuil wrote:
> >
> >
> > On 13/08/15 23:29, Rich Megginson wrote:
> >> On 08/13/2015 12:41 AM, Gilles Dubreuil wrote:
> >>> Hi Matthew,
> >>>
> >>> On 11/08/15 01:14, Rich Megginson wrote:
> >>>> On 08/10/2015 07:46 AM, Matthew Mosesohn wrote:
> >>>>> Sorry to everyone for bringing up this old thread, but it
> seems we may
> >>>>> need more openstackclient/keystone experts to settle this.
> >>>>>
> >>>>> I'm referring to the comments in
> >>>>> https://review.openstack.org/#/c/207873/
> >>>>> Specifically comments from Richard Megginson and Gilles Dubreuil
> >>>>> indicating openstackclient behavior for v3 keystone API.
> >>>>>
> >>>>>
> >>>>> A few items seem to be under dispute:
> >>>>> 1 - Keystone should be able to accept v3 requests at
> >>>>> <http://keystone-server:5000/>http://keystone-server:5000/
> >>>> I don't think so. Keystone requires the version suffix
> "/v2.0" or
> >>>> "/v3".
> >>>>
> >>> Yes, if the public endpoint is set without a version then the
> service
> >>> can deal with either version.
> >>>
> >>> http://paste.openstack.org/raw/412819/
> >>>
> >>> That is not true for the admin endpoint (authentication is
> already done,
> >>> the admin services deals only with tokens), at least for now,
> Keystone
> >>> devs are working on it.
> >>
> >> I thought it worked like this - the openstack cli will infer
> from the
> >> arguments if it should do v2 or v3 auth. In the above case for v3,
> >> since you specify the username/password, osc knows it has to use
> >> password auth (as opposed to token auth), and since all of the
> required
> >> v3 arguments are provided (v3 api version, domains for
> user/project), it
> >> can use v3 password auth. It knows it has to use the
> "/v3/auth/token"
> >> path to get a token.
> >>
> >> Similarly for v2, since it only has username/password, no v3
> api or v3
> >> domain arguments, it knows it has to use v2 password auth. It
> knows it
> >> has to use "/v2.0/token" to get a token.
> >>
> >> With the puppet-keystone code, since it uses TOKEN/URL, osc
> cannot infer
> >> if it can use v2 or v3, so it requires the "/v2.0" or "/v3"
> suffix, and
> >> the api version.
> >>
> >
> > First of my apologies because I confused admin enpdoint with the
> admin
> > service (or whatever that's dubbed).
> >
> > As of Keystone v3 (not the API, the latest version of Keystone which
> > supports both API v2.0 and V3), the OS_AUTH_URL doesn't need the
> > version. That can be effectively any of the endpoints, either
> the main
> > (or public) by default on port 5000 or the admin by default on port
> > 35357, the third one internal pointing to either of the first
> two ones.
> >
> > This is a behavior at Keystone level not at the OSC. I mean OSC will
> > just reflect the http-api behavior.
> >
> > Now the admin service, the one needed for the OS-TOKEN (used for
> > bootstrapping) needs a URL (OS_URL) with a version to work.
> >
> > ATM, the issue with puppet keystone is that endpoints, OS_URL and
> > OS_AUTH_URL are walking on each others.
> >
> >
>
> My latest update on this one, is that I found out while testing beaker
> which is using OSC 1.0.3 is not handling OS_AUTH_URL properly.
>
> I had been testing with OSC 1.5.1 and now latest 1.6.1 from Dolerean
> repo, where the version-less URLs are working, but not with OSC 1.0.1:
>
> ----------------------
>
> # cat openrc
> export OS_AUTH_URL=http://127.0.0.1:5000
> export OS_USERNAME=adminv3
> export OS_PASSWORD=testing
> export OS_PROJECT_NAME=openstackv3
> export OS_USER_DOMAIN_NAME=admin_domain
> export OS_PROJECT_DOMAIN_NAME=admin_domain
> export OS_IDENTITY_API_VERSION="3"
>
> # openstack endpoint list -f csv
> "ID","Region","Service Name","Service
> Type","Enabled","Interface","URL"
> "87b7db1b23df487bb4ec96de5aa3c271","RegionOne","keystone","identity",True,"internal","http://127.0.0.1:5000"
> "d9b345109d8a4320ac0dd832d2532cce","RegionOne","keystone","identity",True,"admin","http://127.0.0.1:35357"
> "f3a579a64f0241ef9aef3dc983e0fd4a","RegionOne","keystone","identity",True,"public","http://127.0.0.1:5000"
>
> ----------------------
>
> # cat openrc_v2
> export OS_AUTH_URL=http://[::1]:5000
> export OS_USERNAME=admin
> export OS_PASSWORD=a_big_secret
> export OS_TENANT_NAME=openstack
>
> # openstack endpoint list -f csv --long
> "ID","Region","Service Name","Service
> Type","PublicURL","AdminURL","InternalURL"
> "cf8825c44bd041109d5c7438ba9db8f3","RegionOne","keystone","identity","http://127.0.0.1:5000","http://127.0.0.1:35357","http://127.0.0.1:5000"
>
I don't understand. Is this output supposed to show a problem? Looks
like it is working.
What is the problem? What is the difference of behavior between OSC
1.0.x and OSC 1.5.x?
>
>
>
>
> >>>
> >>>>> 2 - openstackclient should be able to interpret v3 requests
> and append
> >>>>> "v3/" to OS_AUTH_URL=http://keystone-server.5000/ or rewrite
> the URL
> >>>>> if it is set as
> >>>>>
> OS_AUTH_URL=<http://keystone-server.5000/>http://keystone-server.5000/
> >>>> It does, if it can determine from the given authentication
> arguments if
> >>>> it can do v3 or v2.0.
> >>>>
> >>> It effectively does, again, assuming the path doesn't contain
> a version
> >>> number (x.x.x.x:5000)
> >>>
> >>>>> 3 - All deployments require /etc/keystone/keystone.conf with
> a token
> >>>>> (and not simply use openrc for creating additional
> endpoints, users,
> >>>>> etc beyond keystone itself and an admin user)
> >>>> No. What I said about this issue was "Most people using
> >>>> puppet-keystone, and realizing Keystone resources on nodes
> that are not
> >>>> the Keystone node, put a /etc/keystone/keystone.conf on that
> node with
> >>>> the admin_token in it."
> >>>>
> >>>> That doesn't mean the deployment requires
> /etc/keystone/keystone.conf.
> >>>> It should be possible to realize Keystone resources on
> non-Keystone
> >>>> nodes by using ENV or openrc or other means.
> >>>>
> >>> Agreed. Also keystone.conf is used only to bootstrap keystone
> >>> installation and create admin users, etc.
> >>>
> >>>
> >>>>> I believe it should be possible to set v2.0 keystone
> OS_AUTH_URL in
> >>>>> openrc and puppet-keystone + puppet-openstacklib should be
> able to
> >>>>> make v3 requests sensibly by manipulating the URL.
> >>>> Yes. Because for the puppet-keystone resource providers,
> they are coded
> >>>> to a specific version of the api, and therefore need to be
> able to
> >>>> set/override the OS_IDENTITY_API_VERSION and the version
> suffix in
> >>>> the URL.
> >>>>
> >>> No. To support V2 and V3, the OS_AUTH_URL must not contain any
> version
> >>> in order.
> >>>
> >>> The less we deal with the version number the better!
> >>>
> >>>>> Additionally, creating endpoints/users/roles shouldbe
> possible via
> >>>>> openrc alone.
> >>>> Yes.
> >>>>
> >>> Yes, the openrc variables are used, if not available then the
> service
> >>> token from the keystone.conf is used.
> >>>
> >>>>> It's not possible to write single node tests that can
> demonstrate this
> >>>>> functionality, which is why it probably went undetected for
> so long.
> >>>> And since this is supported, we need tests for this.
> >>> I'm not sure what's the issue besides the fact keystone_puppet
> doesn't
> >>> generate a RC file once the admin user has been created. That
> is left to
> >>> be done by the composition layer. Although we might want to
> integrate
> >>> that.
> >>>
> >>> If that issue persists, assuming the AUTH_URL is free for a
> version
> >>> number and having an openrc in place, we're going to need a
> bug number
> >>> to track the investigation.
> >>>
> >>>>> If anyone can speak up on these items, it could help
> influence the
> >>>>> outcome of this patch.
> >>>>>
> >>>>> Thank you for your time.
> >>>>>
> >>>>> Best Regards,
> >>>>> Matthew Mosesohn
> >>>
> >>> Thanks,
> >>> Gilles
> >>>
> >>>>> On Fri, Jul 31, 2015 at 6:32 PM, Rich Megginson
> <rmeggins at redhat.com <mailto:rmeggins at redhat.com>
> >>>>> <mailto:rmeggins at redhat.com <mailto:rmeggins at redhat.com>>>
> wrote:
> >>>>>
> >>>>> On 07/31/2015 07:18 AM, Matthew Mosesohn wrote:
> >>>>>
> >>>>> Jesse, thanks for raising this. Like you, I should
> just track
> >>>>> upstream
> >>>>> and wait for full V3 support.
> >>>>>
> >>>>> I've taken the quickest approach and written fixes to
> >>>>> puppet-openstacklib and puppet-keystone:
> >>>>> https://review.openstack.org/#/c/207873/
> >>>>> https://review.openstack.org/#/c/207890/
> >>>>>
> >>>>> and again to Fuel-Library:
> >>>>> https://review.openstack.org/#/c/207548/1
> >>>>>
> >>>>> I greatly appreciate the quick support from the
> community to
> >>>>> find an
> >>>>> appropriate solution. Looks like I'm just using a weird
> >>>>> edge case
> >>>>> where we're creating users on a separate node from
> where
> >>>>> keystone is
> >>>>> installed and it never got thoroughly tested, but
> I'm happy
> >>>>> to fix
> >>>>> bugs where I can.
> >>>>>
> >>>>>
> >>>>> Most puppet deployments either realize all keystone
> resources on
> >>>>> the keystone node, or drop an
> /etc/keystone/keystone.conf with
> >>>>> admin token onto non-keystone nodes where additional
> keystone
> >>>>> resources need to be realized.
> >>>>>
> >>>>>
> >>>>>
> >>>>> -Matthew
> >>>>>
> >>>>> On Fri, Jul 31, 2015 at 3:54 PM, Jesse Pretorius
> >>>>> <jesse.pretorius at gmail.com
> <mailto:jesse.pretorius at gmail.com>
> <mailto:jesse.pretorius at gmail.com <mailto:jesse.pretorius at gmail.com>>>
> >>>>> wrote:
> >>>>>
> >>>>> With regards to converting all services to use
> Keystone v3
> >>>>> endpoints, note
> >>>>> the following:
> >>>>>
> >>>>> 1) swift-dispersion currently does not support
> consuming
> >>>>> Keystone v3
> >>>>> endpoints [1]. There is a patch merged to
> master [2] to
> >>>>> fix that, but a
> >>>>> backport to kilo is yet to be done.
> >>>>> 2) Each type (internal, admin, public) of
> endpoint created
> >>>>> with the Keystone
> >>>>> v3 API has its own unique id, unlike with the
> v2 API where
> >>>>> they're all
> >>>>> created with a single ID. This results in the
> keystone
> >>>>> client being unable
> >>>>> to read the catalog created via the v3 API when
> querying
> >>>>> via the v2 API. The
> >>>>> solution is to use the openstack client and to
> use the v3
> >>>>> API but this
> >>>>> obviously needs to be noted for upgrade impact and
> >>>>> operators.
> >>>>> 3) When glance is setup to use swift as a back-end,
> >>>>> glance_store is unable
> >>>>> to authenticate to swift when the endpoint it
> uses is a v3
> >>>>> endpoint. There
> >>>>> is a review to master in progress [3] to fix
> this which is
> >>>>> unlikely to make
> >>>>> it into kilo.
> >>>>>
> >>>>> We (the openstack-ansible/os-ansible-deployment
> project)
> >>>>> are tracking these
> >>>>> issues and doing tests to figure out all the
> bits. These
> >>>>> are the bugs we've
> >>>>> hit so far. Also note that there is a WIP patch
> to gate
> >>>>> purely on Keystone
> >>>>> v3 API's which is planned to become voting
> (hopefully) by
> >>>>> the end of this
> >>>>> cycle.
> >>>>>
> >>>>> [1] https://bugs.launchpad.net/swift/+bug/1468374
> >>>>> [2] https://review.openstack.org/195131
> >>>>> [3] https://review.openstack.org/193422
> >>>>>
> >>>>>
> >>>>>
> __________________________________________________________________________
> >>>>>
> >>>>> OpenStack Development Mailing List (not for usage
> >>>>> questions)
> >>>>> Unsubscribe:
> >>>>>
> >>>>>
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> >>>>>
> >>>>>
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> >>>>>
> >>>>>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>>>
> >>>>>
> >>>>>
> __________________________________________________________________________
> >>>>>
> >>>>> OpenStack Development Mailing List (not for usage
> questions)
> >>>>> Unsubscribe:
> >>>>>
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> >>>>>
> >>>>>
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> >>>>>
> >>>>>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> __________________________________________________________________________
> >>>>>
> >>>>> OpenStack Development Mailing List (not for usage
> questions)
> >>>>> Unsubscribe:
> >>>>>
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> >>>>>
> >>>>>
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> >>>>>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> __________________________________________________________________________
> >>>>>
> >>>>> OpenStack Development Mailing List (not for usage questions)
> >>>>> Unsubscribe:
> >>>>>
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> >>>>>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>>
> >>>>
> >>>>
> __________________________________________________________________________
> >>>>
> >>>> OpenStack Development Mailing List (not for usage questions)
> >>>> Unsubscribe:
> >>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>>
> >>>
> __________________________________________________________________________
> >>>
> >>> OpenStack Development Mailing List (not for usage questions)
> >>> Unsubscribe:
> >>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >>
> >>
> __________________________________________________________________________
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >
> __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150814/e8073358/attachment.html>
More information about the OpenStack-dev
mailing list