[openstack-dev] [ironic] using keystone right - catalog, endpoints, tokens and noauth

Pavlo Shchelokovskyy pshchelokovskyy at mirantis.com
Thu Jun 1 13:01:07 UTC 2017

Hi all,

thanks Monty for the feedback. I've started this set of patches in ironic
[0] (very WiP currently), and would really like some eyes on them (not
right now, as I plan to rewrite those basing on this conversation :) but
soon I hope). Also I have some questions/comments inline:

On Thu, May 25, 2017 at 12:36 AM, Monty Taylor <mordred at inaugust.com> wrote:

> On 05/24/2017 12:51 PM, Eric Fried wrote:
>> Pavlo-
>>         There's a blueprint [1] whereby we're trying to address a bunch of
>> these same concerns in nova.  You can see the first part in action here
>> [2].  However, it has become clear that nova is just one of the many
>> services that would benefit from get_service_url().  With the full
>> support of mordred (let's call it The Full Monty), we've got our sights
>> on moving that method into ksa itself for that purpose.
> Yes - this has started with documenting how to consume Keystone Catalog
> and discovery properly.
> https://review.openstack.org/#/q/topic:version-discovery
> (it's a big stack)
> Once we're good with that - the next step is getting ksa updated to be
> able to handle the end-to-end. It does most of it today, but there are
> enough edgecases it doesn't that you wind up having to do something else,
> like efried just did in nova. The goal is to make that not necessary - and
> so that it's both possible and EASY for everyone to CORRECTLY consume
> catalog and version discovery.
> (more comments inline below)
>         Please have a look at this blueprint and change set.  Let us know
>> if
>> your concerns would be addressed if this were available to you from ksa.
>> [1]
>> https://specs.openstack.org/openstack/nova-specs/specs/pike/
>> approved/use-service-catalog-for-endpoints.html
>> [2] https://review.openstack.org/#/c/458257/
>>                         Thanks,
>>                         efried
>> On 05/24/2017 04:46 AM, Pavlo Shchelokovskyy wrote:
>>> Hi all,
>>> There are several problems or inefficiencies in how we are dealing with
>>> auth to other services. Although it became much better in Newton, some
>>> things are still to be improved and I like to discuss how to tackle
>>> those and my ideas for that.
>>> Keystone endpoints
>>> ===============
>>> Apparently since February-ish DevStack no longer sets up 'internal'
>>> endpoints for most of the services in core devstack [0].
>>> Luckily we were not broken by that right away - although when
>>> discovering a service endpoint from keystone catalog we default to
>>> 'internal' endpoint [1], for most services our devstack plugin still
>>> configures explicit service URL in the corresponding config section, and
>>> thus the service discovery from keystone never takes place (or that code
>>> path is not tested by functional/integration testing).
>>> AFAIK different endpoint types (internal vs public) are still quite used
>>> by deployments (and IMO rightfully so), so we have to continue
>>> supporting that. I propose to take the following actions:
> I agree you should continue supporting it.
> I'm not sure it's important for you to change your defaults ... as long at
> it's possible to consistently set "interface=public" or
> "interface=internal" and have the results be correct, I think that's the
> big win.
> - in our devstack plugin, stop setting up the direct service URLs in
>>> config, always use keystone catalog for discovery
> - in every conf section related to external service add
>>> 'endpoint_type=[internal|public]' option, defaulting to 'internal', with
>>> a warning in option description (and validated on conductor start) that
>>> it will be changed to 'public' in the next release
> efried just added a call to keystoneauth which will register all of the
> appropriate CONF options that are needed to request a service endpoint from
> the catalog - register_adapter_conf_options:
> http://git.openstack.org/cgit/openstack/keystoneauth/tree/ke
> ystoneauth1/loading/__init__.py#n39
> The word "adapter" in this case isn't directly important - but there are
> three general concepts in keystoneauth that relate to how you connect:
> auth
>  - how you authenticate - auth_type, username, password, etc.
> session
>  - how the transport layer connects - certs, timeouts, etc.
> adapter
>  - what base endpoint to mount from the catalog - service_type, interface,
> endpoint_override, api_version

Currently I'm trying to understand how to use adapters for creating
clients. It seems that not all of clients of interest for me support this,
as some ignore 'interface', 'endpoint_override'  etc of the adapter
instance if I'm passing it instead of a session, and always rely on the
same/similar options passed to client. Are there any examples of how to use
them? Also, some clients (e.g. a neutron [1]) already base their
SessionClient on keystoneauth1.adapter.Adapter, so how could one create
such client instance with adapter options loaded from config?

> - use those values from CONF wherever we ask for service URL from
>>> catalog or instantiate client with session.
> - populate these options in our devstack plugin to be 'public'
>>> - in Queens, switch the default to 'public' and use defaults in devstack
>>> plugin, remove warnings.
>>> Unify clients creation
>>> ================
>>> again, in those config sections related to service clients, we have many
>>> options to instantiate clients from (especially glance section, see my
>>> other recent ML about our image service code). Many of those seem to be
>>> from the time when keystone catalog was missing some functionality or
>>> not existing at all, and keystoneauth lib abstracting identity and
>>> client sessions was not there either.
>>> To simplify setup and unify as much code as possible I'd like to propose
>>> the following:
>>> - in each config section for service client add (if missing) a
>>> '<service>_url' option that should point to the API of given service and
>>> will be used *only in noauth mode* when there's no Keystone catalog to
>>> discover the service endpoint from
> I disagre with this one.
> The option exists and is called "endpoint_override" and it skips the
> catalog completely. It will get registered if you use
> register_adapter_conf_options. It's also useful even in auth scenarios
> sometimes.

OK, will use endpoint_override instead, would indeed be more unified

> AND/OR ... (we could do both)
> I do this in os-client-config:
> http://git.openstack.org/cgit/openstack/os-client-config/tre
> e/os_client_config/config.py#n857
> to support ironic clients without keystone (yay bifrost!)
> However, that's a hack. Perhaps what we should do is add an auth plugin to
> keystoneauth called "none" or "noauth". It's the auth plugins that actually
> implement the auth_url support - so a noauth plugin that takes no arguments
> and does nothing - combined with the already existing endpoint_override
> argument could be a path forward that fits into the existing structure
> consistently?

Yes please :) as this is exactly what I was looking for - the NoAuth
auth_plugin. This would really simplify the handling of auth/noauth case in
a single code base. But again, for now it seems I would need to resort to
instantiating both session and adapter from config, and passing session and
explicit adapter attributes to clients directly.

> - in the code creating service clients, always create a keystoneauth
>>> session from config sections, using appropriate keystoneauth identity
>>> plugin - 'token_endpoint' with fake token <service>_url for noauth mode,
>>> 'password' for service user client, 'token' when using a token from
>>> incoming request. The latter will have a benefit to make it possible for
>>> the session to reauth itself when user token is about to expire, but
>>> might require changes in some public methods to pass in the full
>>> task.context instead of just token
> Yes. There is actually a thing in keystoneauth called
> ServiceTokenAuthWrapper
> http://git.openstack.org/cgit/openstack/keystoneauth/tree/ke
> ystoneauth1/service_token.py
> which is used in Nova for cases where that is important.

This is another point I'd have to wrap my head around. Where is the
user_auth coming from in ServiceToken? Can we somehow instantiate a
user_auth from oslo_context.RequestContext or its subclasses? Preferably by
some base method of RequestContext or from keystoneauth loading. I see an
example in Nova, but am still puzzled about user_auth.

> - always create clients from sessions. Although AFAIK all clients ironic
>>> uses already support this, some in ironic code (e.g. glance) still
>>> always create a client from token and endpoint directly.
> YES DEAR GOD YES this is very important.
> - deprecate some options explicitly registered by ironic in those
>>> sections that are becoming redundant - including those that relate to
>>> HTTP session settings (like timeout, retries, SSL certs and settings) as
>>> those will be used from options registered by keystoneauth Session, and
>>> those multiple options that piece together a single service URL.
> Yes. this is very great. As I mentioned earlier, "session" options from
> ksa cover this- so register_session_conf_options will take care of these.
> This will decrease the complexity of service client-related code and
>>> will make configuring those cleaner.
>>> Of course all of this has to be done minding proper deprecation process,
>>> although that might complicate things (as usual :/).
> Nice things are never easy. :)
> Legacy auth
>>> =========
>>> Probably not worth specific mention, but we implemented a proper
>>> keystoneauth-based loading of client auth options back in Newton almost
>>> a year ago, so the code attempting to load auth for clients in a
>>> deprecated way from "[keystone_authtoken]" section can be safely removed
>>> already.
>>> As always, I'm eager to hear your comments.
> Thank you for writing this up - it's very well timed with the other work
> going on related to catalog and version discovery. Please let me know if I
> can help in any way or if anything is unclear.
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



Dr. Pavlo Shchelokovskyy
Senior Software Engineer
Mirantis Inc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20170601/2701106a/attachment.html>

More information about the OpenStack-dev mailing list