Open Stack

Travis,

That said, I can see a few ways that we could use the same REST decorator
> code and provide direct access to the API.  We’d simply provide a class
> where the url_regex maps to the desired path and gives direct passthrough.
> Maybe that kind of passthrough could always be provided for ease of
> customization / extensibility and additional methods with wrappers provided
> when necessary.



I completely agree on this. We can use the REST decorator to handle either
some really specific cases, or to handle some features, like pagination, in
a general way for all entities, if possible. What I argued against was that
it is unnecessary to have duplicate code JS->REST->APIClient, if we can
call directly JS->(auth wrapper)->APIClient. Also, there are some examples
- where the middleware wrapper hides some functionality, or makes some
unneeded processing, that we may completely skip.

In the given example - we don't need the (images, has_prev, has_more)
return value. What we really need is the list of images, sliced based on
the request.GET parameters and the config.API_RESULT_LIMIT. Then - the
whole check for has_prev, has_more should be done in the client. Currently
this processing was done at the server, as it was needed by the Django
rendering engine. Now it is not. So I am basically talking on
simplification of the middleware layer as much as possible, and moving the
presentation logic into the JS Client.


Also, to answer the comment of Thai - there is a lot of work that the
server will still do - like the translation - I guess we should load the
angular templates from the server with applied translation rather than
putting them into plain js files. I'm not sure what are the best options
here. But still - there is a lot of unneeded code currently in the
openstack_dashboard/api/*.py files.

So I guess the current approach with Django-REST might fit our needs. We
just have to look over each /api/ file in greater detail and to remove the
code that will better work on the client.

Let's move the discussion in Gerrit, and discuss the api wrapper proposed
by Richard. I believe we are on the same page now, I just needed to clarify
for myself that we are not going to just replace the Django with REST, but
we want to make Horizon a really flexible and powerful application.


On Sat, Dec 13, 2014 at 1:09 AM, Tripp, Travis S <travis.tripp at hp.com>
wrote:
>
> Tihomir,
>
> Today I added one glance call based on Richard’s decorator pattern[1] and
> started to play with incorporating some of your ideas. Please note, I only
> had limited time today.  That is passing the kwargs through to the glance
> client. This was an interesting first choice, because it immediately
> highlighted a concrete example of the horizon glance wrapper
> post-processing still being useful (rather than be a direct pass-through
> with no wrapper). See below. If you have some some concrete code examples
> of your ideas it would be helpful.
>
> [1]
> https://review.openstack.org/#/c/141273/2/openstack_dashboard/api/rest/glance.py
>
> With the patch, basically, you can call the following and all of the GET
> parameters get passed directly through to the horizon glance client and you
> get results back as expected.
>
>
> http://localhost:8002/api/glance/images/?sort_dir=desc&sort_key=created_at&paginate=True&marker=bb2cfb1c-2234-4f54-aec5-b4916fe2d747
>
> If you pass in an incorrect sort_key, the glance client returns the
> following error message which propagates back to the REST caller as an
> error with the message:
>
> "sort_key must be one of the following: name, status, container_format,
> disk_format, size, id, created_at, updated_at."
>
> This is done by passing **request.GET.dict() through.
>
> Please note, that if you try this (with POSTMAN, for example), you need to
> set the header of X-Requested-With = XMLHttpRequest
>
> So, what issues did it immediately call out with directly invoking the
> client?
>
> The python-glanceclient internally handles pagination by returning a
> generator.  Each iteration on the generator will handle making a request
> for the next page of data. If you were to just do something like return
> list(image_generator) to serialize it back out to the caller, it would
> actually end up making a call back to the server X times to fetch all pages
> before serializing back (thereby not really paginating). The horizon glance
> client wrapper today handles this by using islice intelligently along with
> honoring the API_RESULT_LIMIT setting in Horizon. So, this gives a direct
> example of where the wrapper does something that a direct passthrough to
> the client would not allow.
>
> That said, I can see a few ways that we could use the same REST decorator
> code and provide direct access to the API.  We’d simply provide a class
> where the url_regex maps to the desired path and gives direct passthrough.
> Maybe that kind of passthrough could always be provided for ease of
> customization / extensibility and additional methods with wrappers provided
> when necessary.  I need to leave for today, so can’t actually try that out
> at the moment.
>
> Thanks,
> Travis
>
> From: Thai Q Tran <tqtran at us.ibm.com<mailto:tqtran at us.ibm.com>>
> Reply-To: OpenStack List <openstack-dev at lists.openstack.org<mailto:
> openstack-dev at lists.openstack.org>>
> Date: Friday, December 12, 2014 at 11:05 AM
> To: OpenStack List <openstack-dev at lists.openstack.org<mailto:
> openstack-dev at lists.openstack.org>>
> Subject: Re: [openstack-dev] [horizon] REST and Django
>
>
> In your previous example, you are posting to a certain URL (i.e.
> /keystone/{ver:=x.0}/{method:=update}).
> <client: POST /keystone/{ver:=x.0}/{method:=update}> => <middleware: just
> forward to clients[ver].getattr("method")(**kwargs)> => <keystone: update>
>
> Correct me if I'm wrong, but it looks like you have a unique URL for each
> /service/version/method.
> I fail to see how that is different than what we have today? Is there a
> view for each service? each version?
>
> Let's say for argument sake that you have a single view that takes care of
> all URL routing. All requests pass through this view and contain a JSON
> that contains instruction on which API to invoke and what parameters to
> pass.
> And lets also say that you wrote some code that uses reflection to map the
> JSON to an action. What you end up with is a client-centric application,
> where all of the logic resides client-side. If there are things we want to
> accomplish server-side, it will be extremely hard to pull off. Things like
> caching, websocket, aggregation, batch actions, translation, etc.... What
> you end up with is a client with no help from the server.
>
> Obviously the other extreme is what we have today, where we do everything
> server-side and only using client-side for binding events. I personally
> prefer a more balance approach where we can leverage both the server and
> client. There are things that client can do well, and there are things that
> server can do well. Going the RPC way restrict us to just client
> technologies and may hamper any additional future functionalities we want
> to bring server-side. In other words, using REST over RPC gives us the
> opportunity to use server-side technologies to help solve problems should
> the need for it arises.
>
> I would also argue that the REST approach is NOT what we have today. What
> we have today is a static webpage that is generated server-side, where API
> is hidden from the client. What we end up with using the REST approach is a
> dynamic webpage generated client-side, two very different things. We have
> essentially striped out the rendering logic from Django templating and
> replaced it with Angular.
>
>
> -----Tihomir Trifonov <t.trifonov at gmail.com<mailto:t.trifonov at gmail.com>>
> wrote: -----
> To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org
> >>
> From: Tihomir Trifonov <t.trifonov at gmail.com<mailto:t.trifonov at gmail.com>>
> Date: 12/12/2014 04:53AM
> Subject: Re: [openstack-dev] [horizon] REST and Django
>
> Here's an example: Admin user Joe has an Domain open and stares at it for
> 15 minutes while he updates the description. Admin user Bob is asked to go
> ahead and enable it. He opens the record, edits it, and then saves it. Joe
> finished perfecting the description and saves it. Doing this action would
> mean that the Domain is enabled and the description gets updated. Last man
> in still wins if he updates the same fields, but if they update different
> fields then both of their changes will take affect without them stomping on
> each other. Whether that is good or bad may depend on the situation…
>
>
> That's a great example. I believe that all of the Openstack APIs support
> PATCH updates of arbitrary fields. This way - the frontend(AngularJS) can
> detect which fields are being modified, and to submit only these fields for
> update. If we however use a form with POST, although we will load the
> object before updating it, the middleware cannot find which fields are
> actually modified, and will update them all, which is more likely what PUT
> should do. Thus having full control in the frontend part, we can submit
> only changed fields. If however a service API doesn't support PATCH, it is
> actually a problem in the API and not in the client...
>
>
>
> The service API documentation almost always lags (although, helped by
> specs now) and the service team takes on the burden of exposing a
> programmatic way to access the API.  This is tested and easily consumable
> via the python clients, which removes some guesswork from using the service.
>
> True. But what if the service team modifies a method signature from let's
> say:
>
> def add_something(self, request,
> ​ field1, field2):
>
>
> to
>
> def add_something(self, request,
> ​ field1, field2, field3):
>
>
> and in the middleware we have the old signature:
>
> ​def add_something(self, request,
> ​ field1, field2):
>
> we still need to modify the middleware to add the new field. If however
> the middleware is transparent and just passes **kwargs, it will pass
> through whatever the frontend sends. So we just need to update the
> frontend, which can be done using custom views, and not necessary going
> through an upstream change. My point is why do we need to hide some
> features of the backend service API behind a "firewall" what the middleware
> in fact is?
>
>
>
>
>
>
>
> On Fri, Dec 12, 2014 at 8:08 AM, Tripp, Travis S <travis.tripp at hp.com
> <mailto:travis.tripp at hp.com>> wrote:
> I just re-read and I apologize for the hastily written email I previously
> sent. I’ll try to salvage it with a bit of a revision below (please ignore
> the previous email).
>
> On 12/11/14, 7:02 PM, "Tripp, Travis S" <travis.tripp at hp.com<mailto:
> travis.tripp at hp.com>> wrote
> (REVISED):
>
> >Tihomir,
> >
> >Your comments in the patch were very helpful for me to understand your
> >concerns about the ease of customizing without requiring upstream
> >changes. It also reminded me that I’ve also previously questioned the
> >python middleman.
> >
> >However, here are a couple of bullet points for Devil’s Advocate
> >consideration.
> >
> >
> >  *   Will we take on auto-discovery of API extensions in two spots
> >(python for legacy and JS for new)?
> >  *   The Horizon team will have to keep an even closer eye on every
> >single project and be ready to react if there are changes to the API that
> >break things. Right now in Glance, for example, they are working on some
> >fixes to the v2 API (soon to become v2.3) that will allow them to
> >deprecate v1 somewhat transparently to users of the client library.
> >  *   The service API documentation almost always lags (although, helped
> >by specs now) and the service team takes on the burden of exposing a
> >programmatic way to access the API.  This is tested and easily consumable
> >via the python clients, which removes some guesswork from using the
> >service.
> >  *   This is going to be an incremental approach with legacy support
> >requirements anyway.  So, incorporating python side changes won’t just go
> >away.
> >  *   Which approach would be better if we introduce a server side
> >caching mechanism or a new source of data such as elastic search to
> >improve performance? Would the client side code have to be changed
> >dramatically to take advantage of those improvements or could it be done
> >transparently on the server side if we own the exposed API?
> >
> >I’m not sure I fully understood your example about Cinder.  Was it the
> >cinder client that held up delivery of horizon support, the cinder API or
> >both?  If the API isn’t in, then it would hold up delivery of the feature
> >in any case. There still would be timing pressures to react and build a
> >new view that supports it. For customization, with Richard’s approach new
> >views could be supported by just dropping in a new REST API decorated
> >module with the APIs you want, including direct pass through support if
> >desired to new APIs. Downstream customizations / Upstream changes to
> >views seem a bit like a bit of a related, but different issue to me as
> >long as their is an easy way to drop in new API support.
> >
> >Finally, regarding the client making two calls to do an update:
> >
> >​>>Do we really need the lines:​
> >
> >>> project = api.keystone.tenant_get(request, id)
> >>> kwargs = _tenant_kwargs_from_DATA(request.DATA, enabled=None)
> >​
> >I agree that if you already have all the data it may be bad to have to do
> >another call. I do think there is room for discussing the reasoning,
> >though.
> >As far as I can tell, they do this so that if you are updating an entity,
> >you have to be very specific about the fields you are changing. I
> >actually see this as potentially a protectionary measure against data
> >loss and sometimes a very nice to have feature. It perhaps was intended
> >to *help* guard against race conditions (no locking and no transactions
> >with many users simultaneously accessing the data).
> >
> >Here's an example: Admin user Joe has a Domain open and stares at it for
> >15 minutes while he updates just the description. Admin user Bob is asked
> >to go ahead and enable it. He opens the record, edits it, and then saves
> >it. Joe finished perfecting the description and saves it. They could in
> >effect both edit the same domain independently. Last man in still wins if
> >he updates the same fields, but if they update different fields then both
> >of their changes will take affect without them stomping on each other. Or
> >maybe it is intended to encourage client users to compare their current
> >and previous to see if they should issue a warning if the data changed
> >between getting and updating the data. Or maybe like you said, it is just
> >overhead API calls.
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>


-- 
Regards,
Tihomir Trifonov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20141215/0db685ac/attachment.html>