Open Stack

On Thu, Oct 20, 2016 at 02:01:07AM -0500, Monty Taylor wrote:
> On 10/19/2016 12:07 PM, Matt Riedemann wrote:
> > On 10/19/2016 10:32 AM, Brian Curtin wrote:
> >> I'm currently facing what looks more and more like an impossible
> >> problem in determining the root of each service on a given cloud. It
> >> is apparently a free-for-all in how endpoints can be structured, and I
> >> think we're out of ways to approach it that catch all of the ways that
> >> all people can think of.
> >>
> >> In openstacksdk, we can no longer use the service catalog for
> >> determining each service's endpoints. Among other things, this is due
> >> to a combination of some versions of some services not actually being
> >> listed, and with things heading the direction of version-less services
> >> anyway. Recently we changed to using the service catalog as a pointer
> >> to where services live and then try to find the root of that service
> >> by stripping the path down and making some extra requests on startup
> >> to find what's offered. Despite a few initial snags, this now works
> >> reasonably well in a majority of cases.
> >>
> >> We have seen endpoints structured in the following ways:
> >>  A. subdomains, e.g., https://service.cloud.com/v2
> >>  B. paths, e.g., https://cloud.com/service/v2 (sometimes there are
> >> more paths in between the root and /service/)
> >>  C. service-specific ports, e.g., https://cloud.com:1234/v2
> >>  D. both A and B plus ports
> >>
> >> Within all of these, we can find the root of the given service just
> >> fine. We split the path and build successively longer paths starting
> >> from the root. In the above examples, we need to hit the path just
> >> short of the /v2, so in B it actually takes two requests as we'd make
> >> one to cloud.com which fails, but then a second one to
> >> cloud.com/service gives us what we need.
> >>
> >> However, another case came up: the root of all endpoints is itself
> >> another service. That makes it look like this:
> >>
> >>  E. https://cloud.com:9999/service/v2
> >>  F. https://cloud.com:9999/otherservice
> >>
> >> In this case, https://cloud.com:9999 is keystone, so trying to get E's
> >> base by going from the root and outward will give me a versions
> >> response I can parse properly, but it points to keystone. We then end
> >> up building requests for 'service' that go to keystone endpoints and
> >> end up failing. We're doing this using itertools.accumulate on the
> >> path fragments, so you might think 'just throw it through
> >> `reversed()`' and go the other way. If we do that, we'll also get a
> >> versions response that we can parse, but it's the v2 specific info,
> >> not all available versions.
> >>
> >> So now that we can't reliably go from the left, and we definitely
> >> can't go from the right, how about the middle?
> >>
> >> This sounds ridiculous, and if it sounds familiar it's because they
> >> devise a "middle out" algorithm on the show Silicon Valley, but in
> >> most cases it'd actually work. In E above, it'd be fine. However,
> >> depending on the number of path fragments and which direction we chose
> >> to move first, we'd sometimes hit either a version-specific response
> >> or another service's response, so it's not reliable.
> >>
> >> Ultimately, I would like to know how something like this can be solved.
> >>
> >> 1. Is there any reliable, functional, and accurate programmatic way to
> >> get the versions and endpoints that all services on a cloud offer?
> >>
> >> 2. Are there any guidelines, rules, expectations, or other
> >> documentation on how services can be installed and their endpoints
> >> structured that are helpful to people build apps that use them, not in
> >> those trying to install and operate them? I've looked around a few
> >> times and found nothing useful. A lot of what I've found has
> >> referenced suggestions for operators setting them up behind various
> >> load balancing tools.
> >>
> >> 3. If 1 and 2 won't actually help me solve this, do you have any other
> >> suggestions that will? We already go left, right, and middle of each
> >> URI, so I'm out of directions to go, and we can't go back to the
> >> service catalog.
> >>
> >> Thanks,
> >>
> >> Brian
> >>
> >> __________________________________________________________________________
> >>
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe:
> >> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> > 
> > That's a tricky one. Just yesterday I was looking into how Tempest
> > creates the service clients it uses and lists versions, e.g. for compute:
> > 
> > https://github.com/openstack/tempest/blob/13.0.0/tempest/lib/services/compute/versions_client.py#L27
> > 
> > 
> > I was trying to figure out where that base_url value came from and in
> > the case of Tempest it's from an auth provider class, and I think the
> > services inside that thing are created from this code at some point:
> > 
> > https://github.com/openstack/tempest/blob/13.0.0/tempest/config.py#L1405
> > 
> > So at a basic level that builds the client with config options for the
> > service type (e.g. compute) and endpoint type (e.g. publicURL), so then
> > it can lookup the publicURL for the 'compute' service/endpoint in the
> > service catalog and then start parsing the endpoint URL to do a GET on
> > the root endpoint for compute to get the available API versions.
> > 
> > It doesn't sound like the same case with the SDK though since you don't
> > have a config file telling you what exact things to lookup in the
> > service catalog...which seems like the main issue.
> 
> In os-client-config, we defer all of this to keystoneauth1. Basically,
> we collect service_type, endpoint_type and a pile of other parameters
> (service_name, cert, key, favorite color) and pass them to a few
> different keystoneauth calls to wind up with a
> keystoneauth1.adapter.Adapter, which is a thing which behaves like a
> requests Session but is rooted on a particular endpoint from the catalog
> by name and knows about tokens.
> 
> For instance:
> 
> client = os_client_config.make_rest_client('compute', cloud='vexxhost')
> 

Is there a place for region_name?

> Gets you one of those. With it, you can do:
> 
> client.get('/servers')
> 
> Nova doesn't have more than one major version in reality, so the /
> endpoint isnt' a thing for nova.
> 
> for glance:
> 
> 
> client = os_client_config.make_rest_client('image', cloud='vexxhost')
> client.get('/v1/images')
> # or
> client.get('/v2/images')
> 
> you can, with glance, typically do:
> 
> client.get('/')
> 
> and get the version endpoint.

I know some services "hide" this version endpoint behind authentication
middleware (or somthing similar). In other words, you have to
authenticate to keystone before you can get the version info.

Can we please standardize that?
> 
> In case you're wondering, the output of that last line is:
> 
> {u'versions': [{u'status': u'CURRENT', u'id': u'v2.3', u'links':
> [{u'href': u'http://image-ca-ymq-1.vexxhost.net/v2/', u'rel':
> u'self'}]}, {u'status': u'SUPPORTED', u'id': u'v2.2', u'links':
> [{u'href': u'http://image-ca-ymq-1.vexxhost.net/v2/', u'rel':
> u'self'}]}, {u'status': u'SUPPORTED', u'id': u'v2.1', u'links':
> [{u'href': u'http://image-ca-ymq-1.vexxhost.net/v2/', u'rel':
> u'self'}]}, {u'status': u'SUPPORTED', u'id': u'v2.0', u'links':
> [{u'href': u'http://image-ca-ymq-1.vexxhost.net/v2/', u'rel':
> u'self'}]}, {u'status': u'SUPPORTED', u'id': u'v1.1', u'links':
> [{u'href': u'http://image-ca-ymq-1.vexxhost.net/v1/', u'rel':
> u'self'}]}, {u'status': u'SUPPORTED', u'id': u'v1.0', u'links':
> [{u'href': u'http://image-ca-ymq-1.vexxhost.net/v1/', u'rel': u'self'}]}]}
> 
> (yes, all of these lines are live)
> 
> Similar with neutron:
> 
> >>> client = os_client_config.make_rest_client('network', cloud='vexxhost')
> >>> client.get('/').json()
> {u'versions': [{u'status': u'CURRENT', u'id': u'v2.0', u'links':
> [{u'href': u'http://network-ca-ymq-1.vexxhost.net/v2.0', u'rel':
> u'self'}]}]}
> 
> Volumes is more like nova:
> 
> >>> client = os_client_config.make_rest_client('volume', cloud='vexxhost')
> >>> client.get('/volumes').json()
> {u'volumes': []}
> 
> with get('/') throwing a 404.
>