Open Stack

It should be made clear that this is only for data getters and setters, and
not for RPC. I can imagine someone going: "oh, I have an instance object
that I can set the state on, why don't I just slip a 'reboot()' method in
here, muhaha"

Here's my imagining of how it might work. I'm thinking mostly in the
context of cells, but that kind of pushes over to RPC too.

*History according to matiu:*

So in the beginning we sent *dicts of lists of tuples of things* across the
RPC and their content always changed, and it was hard to figure out what
should be in this data when writing tests. Though we have sqlalchemy
models, we sent so much data across at once, and without a well defined
structure.

Then comstud made those beautiful cells *messaging objects* that makes it
much clearer what was going between cells. However it's still a great deal
of (almost identical) code to add a new message.

Now this method you're proposing is a little different, instead of passing
the whole object across the RPC, you allow the program to request the copy
of the object. So all we need to pass over RPC is an ID and an operation to
do with it, then the operating code can request whatever data it needs,
from where ever it needs to get it from.

*Possible example usecase:*
Instance X in cell 2 gets powered down. Cell 2's compute server has to
update its state in the DB of the API server (in a different cell).

In the current setup, it saves the state to its own copy of the DB, then
pushes that up to the top cell via RPC.

Under the new Object Proposal, it could simply:

instance = Instance.get_by_id(context, obj_id)
instance.is_running = False
instance.save()

Whether you wanted to keep a second copy of the instances table in cell 2
is now transparent to the compute code. The object could be smart enough to
update both copies, or there could be a transparent caching layer, where
subsequent calls to get object by id would just use the cache.
*
*
*Pros for the Object Proposal:*

   - It's a nice, intuitive, standardized API for object data access
   - It will merge a lot of code into one way of doing things (sqlalchemy
   api and some of the cells messaging objects).
   - It makes it clear what attributes an object may have, making unit
   tests more thorough
   - It removes all implementation concerns from the users of the object
   - The data type validation is definitely a step in the right direction

*Possible Cons:*

   - Possible performance issues, for example, some code that updates a
   bunch of objects in a list, may not realize that each save() is going
   across 2 amqp servers to reach the DB. This could be worked around by
   adding a 'bulk update wrapper object' and using that (and having it do the
   actual saving at the end). I'm in favor of something like:

with Transaction() as t do:
  for obj_id in ids:
     obj = Obj.get_by_id(obj_id, t)
     obj.foo = 'x'
     obj.save()

Passing the transaction causes the obj to basically queue up all commands
in the transaction, then when the transaction completes, it sends the whole
update code to the appropriate cell or DB backend.

   - The same thing could be done for reading, you do all the reading code
   in the transaction, and it magically makes that code defer until the whole
   transaction is complete, at which point, it gets all the objects you
   accessed in one call, then runs your code.

*Points from the wiki:*

   - Agreement that this approach is generally useful

Definitely a nice thing to have.

   - Agreement that it should (or should not) go into Oslo

Definitely SHOULD go into oslo, I foresee all projects wanting this
eventually.
Kind Regards,
Matthew Sherborne

On Tue, May 7, 2013 at 9:18 AM, Joshua Harlow <harlowja at yahoo-inc.com>wrote:

> Very cool, will be a welcome addition :-)
>
> I was just wondering about the magic rpc part.
>
> It might be controversial but is a decorator the right way to go?
>
> From a glance at other types of RPC frameworks it seems like they pretty
> strongly separate the object and its functions from the thing doing RPC on
> behalf of object. Maybe what you are doing is similar (and hidden behind
> said decorator?). I wonder if instead of a function level decorator that u
> could basically wrap a given object (and its functions) with a proxy
> object and let said proxy object call via RPC. This avoids more decorator
> nastiness?
>
> This seems similar to the xmlrpclib or pyro4 (in concept) which does
> similar automagic conversion.
>
> * http://hg.python.org/cpython/file/8527427914a2/Lib/xmlrpclib.py#l1212 is
> interesting.
> * http://pythonhosted.org/Pyro4/clientcode.html might be similarily
> interesting.
>
> On 5/6/13 1:36 PM, "Dan Smith" <dms at danplanet.com> wrote:
>
> >Hi all,
> >
> >At the summit a few weeks ago, Chris Behrens and I colluded with
> >several other Nova developers to work on a proposal for implementing
> >objects in Nova that would ease some of the pain caused by moving to
> >always passing primitives over RPC. Our goals are:
> >
> >1. Better organize and co-locate data with its implementation
> >2. More intelligently (de-)serialize object data over RPC
> >3. Version the format of the data to assist live upgrade
> >
> >We've been working on some proof-of-concept code to do this, and I've
> >written up a wiki page describing what we're trying to do:
> >
> >  https://wiki.openstack.org/wiki/ObjectProposal
> >
> >We'd appreciate some feedback on the approach, as well as discussion
> >about whether something like this belongs in Oslo so that it can be
> >used by other projects.
> >
> >Thanks!
> >
> >--Dan
> >
> >_______________________________________________
> >OpenStack-dev mailing list
> >OpenStack-dev at lists.openstack.org
> >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130507/0999aede/attachment.html>