Open Stack

On 05/24/2017 12:51 PM, Eric Fried wrote:
> Pavlo-
>
> 	There's a blueprint [1] whereby we're trying to address a bunch of
> these same concerns in nova.  You can see the first part in action here
> [2].  However, it has become clear that nova is just one of the many
> services that would benefit from get_service_url().  With the full
> support of mordred (let's call it The Full Monty), we've got our sights
> on moving that method into ksa itself for that purpose.

Yes - this has started with documenting how to consume Keystone Catalog 
and discovery properly.

https://review.openstack.org/#/q/topic:version-discovery

(it's a big stack)

Once we're good with that - the next step is getting ksa updated to be 
able to handle the end-to-end. It does most of it today, but there are 
enough edgecases it doesn't that you wind up having to do something 
else, like efried just did in nova. The goal is to make that not 
necessary - and so that it's both possible and EASY for everyone to 
CORRECTLY consume catalog and version discovery.

(more comments inline below)

> 	Please have a look at this blueprint and change set.  Let us know if
> your concerns would be addressed if this were available to you from ksa.
>
> [1]
> https://specs.openstack.org/openstack/nova-specs/specs/pike/approved/use-service-catalog-for-endpoints.html
> [2] https://review.openstack.org/#/c/458257/
>
> 			Thanks,
> 			efried
>
> On 05/24/2017 04:46 AM, Pavlo Shchelokovskyy wrote:
>> Hi all,
>>
>> There are several problems or inefficiencies in how we are dealing with
>> auth to other services. Although it became much better in Newton, some
>> things are still to be improved and I like to discuss how to tackle
>> those and my ideas for that.
>>
>> Keystone endpoints
>> ===============
>>
>> Apparently since February-ish DevStack no longer sets up 'internal'
>> endpoints for most of the services in core devstack [0].
>> Luckily we were not broken by that right away - although when
>> discovering a service endpoint from keystone catalog we default to
>> 'internal' endpoint [1], for most services our devstack plugin still
>> configures explicit service URL in the corresponding config section, and
>> thus the service discovery from keystone never takes place (or that code
>> path is not tested by functional/integration testing).
>>
>> AFAIK different endpoint types (internal vs public) are still quite used
>> by deployments (and IMO rightfully so), so we have to continue
>> supporting that. I propose to take the following actions:

I agree you should continue supporting it.

I'm not sure it's important for you to change your defaults ... as long 
at it's possible to consistently set "interface=public" or 
"interface=internal" and have the results be correct, I think that's the 
big win.

>> - in our devstack plugin, stop setting up the direct service URLs in
>> config, always use keystone catalog for discovery

YES

>> - in every conf section related to external service add
>> 'endpoint_type=[internal|public]' option, defaulting to 'internal', with
>> a warning in option description (and validated on conductor start) that
>> it will be changed to 'public' in the next release

efried just added a call to keystoneauth which will register all of the 
appropriate CONF options that are needed to request a service endpoint 
from the catalog - register_adapter_conf_options:

http://git.openstack.org/cgit/openstack/keystoneauth/tree/keystoneauth1/loading/__init__.py#n39

The word "adapter" in this case isn't directly important - but there are 
three general concepts in keystoneauth that relate to how you connect:

auth
  - how you authenticate - auth_type, username, password, etc.
session
  - how the transport layer connects - certs, timeouts, etc.
adapter
  - what base endpoint to mount from the catalog - service_type, 
interface, endpoint_override, api_version

>> - use those values from CONF wherever we ask for service URL from
>> catalog or instantiate client with session.

YES

>> - populate these options in our devstack plugin to be 'public'
>> - in Queens, switch the default to 'public' and use defaults in devstack
>> plugin, remove warnings.
>>
>> Unify clients creation
>> ================
>>
>> again, in those config sections related to service clients, we have many
>> options to instantiate clients from (especially glance section, see my
>> other recent ML about our image service code). Many of those seem to be
>> from the time when keystone catalog was missing some functionality or
>> not existing at all, and keystoneauth lib abstracting identity and
>> client sessions was not there either.
>>
>> To simplify setup and unify as much code as possible I'd like to propose
>> the following:
>>
>> - in each config section for service client add (if missing) a
>> '<service>_url' option that should point to the API of given service and
>> will be used *only in noauth mode* when there's no Keystone catalog to
>> discover the service endpoint from

I disagre with this one.

The option exists and is called "endpoint_override" and it skips the 
catalog completely. It will get registered if you use 
register_adapter_conf_options. It's also useful even in auth scenarios 
sometimes.

AND/OR ... (we could do both)

I do this in os-client-config:

http://git.openstack.org/cgit/openstack/os-client-config/tree/os_client_config/config.py#n857

to support ironic clients without keystone (yay bifrost!)

However, that's a hack. Perhaps what we should do is add an auth plugin 
to keystoneauth called "none" or "noauth". It's the auth plugins that 
actually implement the auth_url support - so a noauth plugin that takes 
no arguments and does nothing - combined with the already existing 
endpoint_override argument could be a path forward that fits into the 
existing structure consistently?

>> - in the code creating service clients, always create a keystoneauth
>> session from config sections, using appropriate keystoneauth identity
>> plugin - 'token_endpoint' with fake token <service>_url for noauth mode,
>> 'password' for service user client, 'token' when using a token from
>> incoming request. The latter will have a benefit to make it possible for
>> the session to reauth itself when user token is about to expire, but
>> might require changes in some public methods to pass in the full
>> task.context instead of just token

Yes. There is actually a thing in keystoneauth called 
ServiceTokenAuthWrapper

http://git.openstack.org/cgit/openstack/keystoneauth/tree/keystoneauth1/service_token.py

which is used in Nova for cases where that is important.

>> - always create clients from sessions. Although AFAIK all clients ironic
>> uses already support this, some in ironic code (e.g. glance) still
>> always create a client from token and endpoint directly.

YES DEAR GOD YES this is very important.

>> - deprecate some options explicitly registered by ironic in those
>> sections that are becoming redundant - including those that relate to
>> HTTP session settings (like timeout, retries, SSL certs and settings) as
>> those will be used from options registered by keystoneauth Session, and
>> those multiple options that piece together a single service URL.

Yes. this is very great. As I mentioned earlier, "session" options from 
ksa cover this- so register_session_conf_options will take care of these.

>> This will decrease the complexity of service client-related code and
>> will make configuring those cleaner.
>>
>> Of course all of this has to be done minding proper deprecation process,
>> although that might complicate things (as usual :/).

Nice things are never easy. :)

>> Legacy auth
>> =========
>>
>> Probably not worth specific mention, but we implemented a proper
>> keystoneauth-based loading of client auth options back in Newton almost
>> a year ago, so the code attempting to load auth for clients in a
>> deprecated way from "[keystone_authtoken]" section can be safely removed
>> already.
>>
>> As always, I'm eager to hear your comments.

Thank you for writing this up - it's very well timed with the other work 
going on related to catalog and version discovery. Please let me know if 
I can help in any way or if anything is unclear.