<div dir="ltr"><div><div><p>It should be made clear that this is only for data getters and setters, and not for RPC. I can imagine someone going: "oh, I have an instance object that I can set the state on, why don't I just slip a 'reboot()' method in here, muhaha"</p>
<p>Here's my imagining of how it might work. I'm thinking mostly in the context of cells, but that kind of pushes over to RPC too.<br></p><p><b>History according to matiu:</b><br></p><p>So in the beginning we sent <i>dicts of lists of tuples of things</i> across the RPC and their content always changed, and it was hard to figure out what should be in this data when writing tests. Though we have sqlalchemy models, we sent so much data across at once, and without a well defined structure.<br>
</p><p>Then comstud made those beautiful cells <i>messaging objects</i> that makes it much clearer what was going between cells. However it's still a great deal of (almost identical) code to add a new message.<br></p>
<p>Now this method you're proposing is a little different, instead of passing the whole object across the RPC, you allow the program to request the copy of the object. So all we need to pass over RPC is an ID and an operation to do with it, then the operating code can request whatever data it needs, from where ever it needs to get it from.</p>
<p><b>Possible example usecase:</b><br></p>Instance X in cell 2 gets powered down. Cell 2's compute server has to update its state in the DB of the API server (in a different cell).<br><br></div><div>In the current setup, it saves the state to its own copy of the DB, then pushes that up to the top cell via RPC.<br>
<br></div><div>Under the new Object Proposal, it could simply:<br></div></div><pre>instance = Instance.get_by_id(context, obj_id)
instance.is_running = False<br>instance.save()<br></pre><div><div>Whether you wanted to keep a second copy of the instances table in cell 2 is now transparent to the compute code. The object could be smart enough to update both copies, or there could be a transparent caching layer, where subsequent calls to get object by id would just use the cache.<br>
</div><div><b><br></b></div><div><b>Pros for the Object Proposal:</b><br><ul><li>It's a nice, intuitive, standardized API for object data access</li><li>It will merge a lot of code into one way of doing things (sqlalchemy api and some of the cells messaging objects).<br>
</li><li>It makes it clear what attributes an object may have, making unit tests more thorough</li><li>It removes all implementation concerns from the users of the object</li><li>The data type validation is definitely a step in the right direction</li>
</ul><p><b>Possible Cons:</b></p><ul><li>Possible performance issues, for example, some code that updates a bunch of objects in a list, may not realize that each save() is going across 2 amqp servers to reach the DB. This could be worked around by adding a 'bulk update wrapper object' and using that (and having it do the actual saving at the end). I'm in favor of something like:</li>
</ul></div><div style="margin-left:80px"><font face="courier new,monospace">with Transaction() as t do:<br></font></div><div style="margin-left:80px"><font face="courier new,monospace"> for obj_id in ids:<br></font></div>
<div style="margin-left:80px"><font face="courier new,monospace"> obj = Obj.get_by_id(obj_id, t)<br></font></div><div style="margin-left:80px"><font face="courier new,monospace"> obj.foo = 'x'<br></font></div>
<div style="margin-left:80px"><font face="courier new,monospace"> obj.save()<br></font></div></div><div style="margin-left:40px"><br></div><div style="margin-left:40px">Passing the transaction causes the obj to basically queue up all commands in the transaction, then when the transaction completes, it sends the whole update code to the appropriate cell or DB backend.<br>
</div><ul><li>The same thing could be done for reading, you do all the reading code in the transaction, and it magically makes that code defer until the whole transaction is complete, at which point, it gets all the objects you accessed in one call, then runs your code.</li>
</ul><p><b>Points from the wiki:</b><br></p><div><div><ul><li> Agreement that this approach is generally useful</li></ul><p>Definitely a nice thing to have.<br></p><ul><li> Agreement that it should (or should not) go into Oslo
</li></ul><p>Definitely SHOULD go into oslo, I foresee all projects wanting this eventually.</p>Kind Regards,<br>Matthew Sherborne<br></div><div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, May 7, 2013 at 9:18 AM, Joshua Harlow <span dir="ltr"><<a href="mailto:harlowja@yahoo-inc.com" target="_blank">harlowja@yahoo-inc.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Very cool, will be a welcome addition :-)<br>
<br>
I was just wondering about the magic rpc part.<br>
<br>
It might be controversial but is a decorator the right way to go?<br>
<br>
>From a glance at other types of RPC frameworks it seems like they pretty<br>
strongly separate the object and its functions from the thing doing RPC on<br>
behalf of object. Maybe what you are doing is similar (and hidden behind<br>
said decorator?). I wonder if instead of a function level decorator that u<br>
could basically wrap a given object (and its functions) with a proxy<br>
object and let said proxy object call via RPC. This avoids more decorator<br>
nastiness?<br>
<br>
This seems similar to the xmlrpclib or pyro4 (in concept) which does<br>
similar automagic conversion.<br>
<br>
* <a href="http://hg.python.org/cpython/file/8527427914a2/Lib/xmlrpclib.py#l1212" target="_blank">http://hg.python.org/cpython/file/8527427914a2/Lib/xmlrpclib.py#l1212</a> is<br>
interesting.<br>
* <a href="http://pythonhosted.org/Pyro4/clientcode.html" target="_blank">http://pythonhosted.org/Pyro4/clientcode.html</a> might be similarily<br>
interesting.<br>
<div class=""><div class="h5"><br>
On 5/6/13 1:36 PM, "Dan Smith" <<a href="mailto:dms@danplanet.com">dms@danplanet.com</a>> wrote:<br>
<br>
>Hi all,<br>
><br>
>At the summit a few weeks ago, Chris Behrens and I colluded with<br>
>several other Nova developers to work on a proposal for implementing<br>
>objects in Nova that would ease some of the pain caused by moving to<br>
>always passing primitives over RPC. Our goals are:<br>
><br>
>1. Better organize and co-locate data with its implementation<br>
>2. More intelligently (de-)serialize object data over RPC<br>
>3. Version the format of the data to assist live upgrade<br>
><br>
>We've been working on some proof-of-concept code to do this, and I've<br>
>written up a wiki page describing what we're trying to do:<br>
><br>
> <a href="https://wiki.openstack.org/wiki/ObjectProposal" target="_blank">https://wiki.openstack.org/wiki/ObjectProposal</a><br>
><br>
>We'd appreciate some feedback on the approach, as well as discussion<br>
>about whether something like this belongs in Oslo so that it can be<br>
>used by other projects.<br>
><br>
>Thanks!<br>
><br>
>--Dan<br>
><br>
>_______________________________________________<br>
>OpenStack-dev mailing list<br>
><a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</div></div></blockquote></div><br></div></div></div></div>