<!DOCTYPE html>
<html>
<head>
<title></title>
</head>
<body><div> </div>
<div> </div>
<div> </div>
<div>On Fri, Apr 22, 2016, at 09:57 PM, Tim Bell wrote:<br></div>
<blockquote type="cite"><div><div>I have reservations on f and g.<br></div>
<div> </div>
<div>On f., We have had a number of discussions in the past about centralising quota (e.g. Boson) and the project teams of the other components wanted to keep the quota contents ‘close’. This can always be reviewed further with them but I would hope for at
least a standard schema structure of tables in each project for the handling of quota.<br></div>
<div> </div>
<div>On g., aren’t all projects now nested projects ? If we have the complexity of handling nested projects sorted out in the common library, is there a reason why a project would not want to support nested projects ?<br></div>
<div> </div>
<div>One other issue is how to do reconcilliation, each project needs to have a mechanism to re-calculate the current allocations and reconcile that with the quota usage. While in an ideal world, this should not be necessary, it would be for the foreseeable
future, especially with a new implementation.<br></div>
</div>
</blockquote><div> </div>
<div>One of the big reasons that Jay and I have been pushing to remove reservations and tracking of quota in a separate place than the resources are actually used, e.g., an instance record in the Nova db, is so that reconciliation is not necessary. For example, if RAM quota usage is simply tracked as sum(instances.memory_mb) then you can be sure that usage is always up to date.<br></div>
<div> </div>
<div> </div>
<blockquote type="cite"><div><div> </div>
<div>Tim<br></div>
<div><div> </div>
</div>
</div>
<div> </div>
<div> </div>
<div style="font-family:Calibri;font-size:12pt;text-align:left;color:black;border-bottom-width:medium;border-bottom-style:none;border-bottom-color:initial;border-left-width:medium;border-left-style:none;border-left-color:initial;padding-bottom:0in;padding-left:0in;padding-right:0in;border-top-color:rgb(181, 196, 223);border-top-width:1pt;border-top-style:solid;border-right-width:medium;border-right-style:none;border-right-color:initial;padding-top:3pt;"><span><b>From: </b>Amrith Kumar <<a href="mailto:amrith@tesora.com">amrith@tesora.com</a>><br> <b>Reply-To: </b>"OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>><br> <b>Date: </b>Friday 22 April 2016 at 06:51<br> <b>To: </b>"OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>><br> <b>Subject: </b>Re: [openstack-dev] More on the topic of DELIMITER, the Quota Management Library proposal</span></div>
<div> </div>
<div> </div>
<blockquote style=""><div><div> </div>
<div lang="EN-US"><div><p><span><span style=""><span class="colour" style="color:rgb(31, 73, 125)"><span class="font" style="font-family:Consolas"><span class="size" style="font-size:11pt">I’ve thought more about Jay’s approach to enforcing quotas and I think we can build on and around it. With that implementation as the basic quota primitive, I think we can
build a quota management API that isn’t dependent on reservations. It does place some burdens on the consuming projects that I had hoped to avoid and these will cause heartburn for some (make sure that you always request resources in a consistent order and
free them in a consistent order being the most obvious).</span></span></span></span></span><br></p><p><span><span style=""><span class="colour" style="color:rgb(31, 73, 125)"><span class="font" style="font-family:Consolas"><span class="size" style="font-size:11pt"> </span></span></span></span></span><br></p><p><span><span style=""><span class="colour" style="color:rgb(31, 73, 125)"><span class="font" style="font-family:Consolas"><span class="size" style="font-size:11pt">If it doesn’t make it harder, I would like to see if we can make the quota API take care of the ordering of requests. i.e. if the quota API is an extension of Jay’s example
and accepts some data structure (dict?) with all the claims that a project wants to make for some operation, and then proceeds to make those claims for the project in the consistent order, I think it would be of some value.</span></span></span></span></span><br></p><p><span><span style=""><span class="colour" style="color:rgb(31, 73, 125)"><span class="font" style="font-family:Consolas"><span class="size" style="font-size:11pt"> </span></span></span></span></span><br></p><p><span><span style=""><span class="colour" style="color:rgb(31, 73, 125)"><span class="font" style="font-family:Consolas"><span class="size" style="font-size:11pt">Beyond that, I’m on board with a-g below,</span></span></span></span></span><br></p><p><span><span style=""><span class="colour" style="color:rgb(31, 73, 125)"><span class="font" style="font-family:Consolas"><span class="size" style="font-size:11pt"> </span></span></span></span></span><br></p><p><span><span style=""><span class="colour" style="color:rgb(31, 73, 125)"><span class="font" style="font-family:Consolas"><span class="size" style="font-size:11pt">-amrith</span></span></span></span></span><br></p><p><span><span style=""><span class="colour" style="color:rgb(31, 73, 125)"><span class="font" style="font-family:Consolas"><span class="size" style="font-size:11pt"> </span></span></span></span></span><br></p><div style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:blue;border-left-width:1.5pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:4pt;"><div><div style="border-right-style:none;border-bottom-style:none;border-left-style:none;border-right-width:initial;border-bottom-width:initial;border-left-width:initial;border-right-color:initial;border-bottom-color:initial;border-left-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-top-style:solid;border-top-color:rgb(225, 225, 225);border-top-width:1pt;padding-top:3pt;padding-right:0in;padding-bottom:0in;padding-left:0in;"><p><span><span style=""><b><span class="font" style="font-family:Calibri, sans-serif"><span class="size" style="font-size:11pt">From:</span></span></b><span class="font" style="font-family:Calibri, sans-serif"><span class="size" style="font-size:11pt"> Vilobh Meshram [<a href="mailto:vilobhmeshram.openstack@gmail.com">mailto:vilobhmeshram.openstack@gmail.com</a>] <br> <b>Sent:</b> Friday, April 22, 2016 4:08 AM<br> <b>To:</b> OpenStack Development Mailing List (not for usage questions) <<a href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>><br> <b>Subject:</b> Re: [openstack-dev] More on the topic of DELIMITER, the Quota Management Library proposal</span></span></span></span></p></div>
</div>
<p><span><span style=""> </span></span><br></p><div><div><p><span><span style=""><span class="font" style="font-family:Georgia, serif">I strongly agree with Jay on the points related to "no reservation" , keeping the interface simple and the role for Delimiter (impose limits on resource consumption and enforce quotas). </span></span></span><br></p></div>
<div><p><span><span style=""> </span></span><br></p></div>
<div><p><span><span style=""><span class="font" style="font-family:Georgia, serif">The point to keep user quota, tenant quotas in Keystone sounds interestring and would need support from Keystone team. We have a Cross project session planned [1] and will definitely bring that
up in that session.</span></span></span><br></p><div><p><span><span style=""> </span></span><br></p></div>
<div><p><span><span style=""><span class="font" style="font-family:Georgia, serif">The main thought with which Delimiter was formed was to enforce resource quota in transaction safe manner and do it in a cross-project conducive manner and it still holds true. Delimiters mission
is to impose limits on resource consumption and enforce quotas in transaction safe manner. Few key aspects of Delimiter are :-</span></span></span><br></p></div>
<div><div><p style=""><span><span style=""><span class="font" style="font-family:Georgia, serif">a. Delimiter will be a new Library and not a Service. Details covered in spec.</span></span></span><br></p><p style=""><span><span style=""><span class="font" style="font-family:Georgia, serif">b. Delimiter's role will be to impose limits on resource consumption.</span></span></span><br></p><p style=""><span><span style=""><span class="font" style="font-family:Georgia, serif">c. Delimiter will not be responsible for rate limiting.</span></span></span><br></p><p style=""><span><span style=""><span class="font" style="font-family:Georgia, serif">d. Delimiter will not maintain data for the resources. Respective projects will take care of keeping, maintaining data for the resources
and resource consumption.</span></span></span><br></p><p style=""><span><span style=""><span class="font" style="font-family:Georgia, serif">e. Delimiter will not have the concept of "reservations". Delimiter will read or update the "actual" resource tables and will not rely
on the "cached" tables. At present, the quota infrastructure in Nova, Cinder and other projects have tables such as reservations, quota_usage, etc which are used as "cached tables" to track re</span></span></span><br></p><p style=""><span><span style=""><span class="font" style="font-family:Georgia, serif">f. Delimiter will fetch the information for project quota, user quota from a centralized place, say Keystone, or if that doesn't materialize
will fetch default quota values from respective service. This information will be cached since it gets updated rarely but read many times.</span></span></span><br></p><p style=""><span><span style=""><span class="font" style="font-family:Georgia, serif">g. Delimiter will take into consideration whether the project is a Flat or Nested and will make the calculations of allocated, available
resources. Nested means project namespace is hierarchical and Flat means project namespace is not hierarchical.</span></span></span><br></p><p style=""><span><span style=""><span class="font" style="font-family:Georgia, serif">-Vilobh</span></span></span><br></p><p style=""><span><span style=""><span class="font" style="font-family:Georgia, serif">[1] <a href="https://www.openstack.org/summit/austin-2016/summit-schedule/events/9492">https://www.openstack.org/summit/austin-2016/summit-schedule/events/9492</a></span></span></span><br></p></div>
</div>
<div><p><span><span style=""> </span></span><br></p><div><p><span><span style=""><span class="font" style="font-family:Georgia, serif">On Thu, Apr 21, 2016 at 11:08 PM, Joshua Harlow <<a href="mailto:harlowja@fastmail.com">harlowja@fastmail.com</a>> wrote:</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif">Since people will be on a plane soon,<br> <br> I threw this together as a example of a quota engine (the zookeeper code does even work, and yes it provides transactional semantics due to the nice abilities of zookeeper znode versions[1] and its inherent consistency model, yippe).<br> <br> <a href="https://gist.github.com/harlowja/e7175c2d76e020a82ae94467a1441d85">https://gist.github.com/harlowja/e7175c2d76e020a82ae94467a1441d85</a><br> <br> Someone else can fill in the db quota engine with a similar/equivalent api if they so dare, ha. Or even feel to say the gist/api above is crap, cause that's ok to, lol.<br> <br> [1] <a href="https://zookeeper.apache.org/doc/r3.1.2/zookeeperProgrammers.html#Data+Access"> https://zookeeper.apache.org/doc/r3.1.2/zookeeperProgrammers.html#Data+Access</a></span></span></span></p><div><div><p><span><span style=""><span class="font" style="font-family:Georgia, serif"><br><br>Amrith Kumar wrote:</span></span></span></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif">Inline below ... thread is too long, will catch you in Austin.</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif">-----Original Message-----<br> From: Jay Pipes [mailto:<a href="mailto:jaypipes@gmail.com">jaypipes@gmail.com</a>]<br> Sent: Thursday, April 21, 2016 8:08 PM<br> To: <a href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a><br> Subject: Re: [openstack-dev] More on the topic of DELIMITER, the Quota<br> Management Library proposal<br> <br> Hmm, where do I start... I think I will just cut to the two primary<br> disagreements I have. And I will top-post because this email is way too<br> big.<br> <br> 1) On serializable isolation level.<br> <br> No, you don't need it at all to prevent races in claiming. Just use a<br> compare-and-update with retries strategy. Proof is here:<br> <br> <a href="https://github.com/jaypipes/placement-bench/blob/master/placement.py#L97-">https://github.com/jaypipes/placement-bench/blob/master/placement.py#L97-</a><br> L142<br> <br> Works great and prevents multiple writers from oversubscribing any<br> resource without relying on any particular isolation level at all.<br> <br> The `generation` field in the inventories table is what allows multiple<br> writers to ensure a consistent view of the data without needing to rely on<br> heavy lock-based semantics and/or RDBMS-specific isolation levels.</span></span></span></p></blockquote><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif"><br>[amrith] this works for what it is doing, we can definitely do this. This will work at any isolation level, yes. I didn't want to go this route because it is going to still require an insert into another table recording what the actual 'thing' is that is claiming
the resource and that insert is going to be in a different transaction and managing those two transactions was what I wanted to avoid. I was hoping to avoid having two tables tracking claims, one showing the currently claimed quota and another holding the
things that claimed that quota. Have to think again whether that is possible.</span></span></span></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif">2) On reservations.<br> <br> The reason I don't believe reservations are necessary to be in a quota<br> library is because reservations add a concept of a time to a claim of some<br> resource. You reserve some resource to be claimed at some point in the<br> future and release those resources at a point further in time.<br> <br> Quota checking doesn't look at what the state of some system will be at<br> some point in the future. It simply returns whether the system *right<br> now* can handle a request *right now* to claim a set of resources.<br> <br> If you want reservation semantics for some resource, that's totally cool,<br> but IMHO, a reservation service should live outside of the service that is<br> actually responsible for providing resources to a consumer.<br> Merging right-now quota checks and future-based reservations into the same<br> library just complicates things unnecessarily IMHO.</span></span></span></p></blockquote><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif"><br>[amrith] extension of the above ...</span></span></span></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif">3) On resizes.<br> <br> Look, I recognize some users see some value in resizing their resources.<br> That's fine. I personally think expand operations are fine, and that<br> shrink operations are really the operations that should be prohibited in<br> the API. But, whatever, I'm fine with resizing of requested resource<br> amounts. My big point is if you don't have a separate table that stores<br> quota_usages and instead only have a single table that stores the actual<br> resource usage records, you don't have to do *any* quota check operations<br> at all upon deletion of a resource. For modifying resource amounts (i.e. a<br> resize) you merely need to change the calculation of requested resource<br> amounts to account for the already-consumed usage amount.<br> <br> Bottom line for me: I really won't support any proposal for a complex<br> library that takes the resource claim process out of the hands of the<br> services that own those resources. The simpler the interface of this<br> library, the better.</span></span></span></p></blockquote><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif"><br>[amrith] my proposal would not but this email thread has got too long. Yes, simpler interface, will catch you in Austin.</span></span></span></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif">Best,<br> -jay<br> <br> On 04/19/2016 09:59 PM, Amrith Kumar wrote:</span></span></span></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif">-----Original Message-----<br> From: Jay Pipes [mailto:<a href="mailto:jaypipes@gmail.com">jaypipes@gmail.com</a>]<br> Sent: Monday, April 18, 2016 2:54 PM<br> To: <a href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a><br> Subject: Re: [openstack-dev] More on the topic of DELIMITER, the<br> Quota Management Library proposal<br> <br> On 04/16/2016 05:51 PM, Amrith Kumar wrote:</span></span></span></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif">If we therefore assume that this will be a Quota Management Library,<br> it is safe to assume that quotas are going to be managed on a<br> per-project basis, where participating projects will use this library.<br> I believe that it stands to reason that any data persistence will<br> have to be in a location decided by the individual project.</span></span></span></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">Depends on what you mean by "any data persistence". If you are<br> referring to the storage of quota values (per user, per tenant,<br> global, etc) I think that should be done by the Keystone service.<br> This data is essentially an attribute of the user or the tenant or the</span></span></span></p></blockquote></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">service endpoint itself (i.e.</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif">global defaults). This data also rarely changes and logically belongs<br> to the service that manages users, tenants, and service endpoints:</span></span></span></p></blockquote></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">Keystone.</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif">If you are referring to the storage of resource usage records, yes,<br> each service project should own that data (and frankly, I don't see a<br> need to persist any quota usage data at all, as I mentioned in a<br> previous reply to Attila).</span></span></span></p></blockquote><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif">[amrith] You make a distinction that I had made implicitly, and it is<br> important to highlight it. Thanks for pointing it out. Yes, I meant<br> both of the above, and as stipulated. Global defaults in keystone<br> (somehow, TBD) and usage records, on a per-service basis.</span></span></span></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif">That may not be a very interesting statement but the corollary is, I<br> think, a very significant statement; it cannot be assumed that the<br> quota management information for all participating projects is in<br> the same database.</span></span></span></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">It cannot be assumed that this information is even in a database at</span></span></span><br></p></blockquote></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">all...</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif">[amrith] I don't follow. If the service in question is to be scalable,<br> I think it stands to reason that there must be some mechanism by which<br> instances of the service can share usage records (as you refer to<br> them, and I like that term). I think it stands to reason that there<br> must be some database, no?</span></span></span></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif">A hypothetical service consuming the Delimiter library provides<br> requesters with some widgets, and wishes to track the widgets that<br> it has provisioned both on a per-user basis, and on the whole. It<br> should therefore multi-tenant and able to track the widgets on a per<br> tenant basis and if required impose limits on the number of widgets<br> that a tenant may consume at a time, during a course of a period of<br> time, and so on.</span></span></span></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">No, this last part is absolutely not what I think quota management<br> should be about.<br> <br> Rate limiting -- i.e. how many requests a particular user can make of<br> an API in a given period of time -- should *not* be handled by<br> OpenStack API services, IMHO. It is the responsibility of the<br> deployer to handle this using off-the-shelf rate-limiting solutions</span></span></span></p></blockquote></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">(open source or proprietary).</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif">Quotas should only be about the hard limit of different types of<br> resources that a user or group of users can consume at a given time.</span></span></span></p></blockquote><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif">[amrith] OK, good point. Agreed as stipulated.</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif">Such a hypothetical service may also consume resources from other<br> services that it wishes to track, and impose limits on.</span></span></span></p></blockquote><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif">Yes, absolutely agreed.</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif">It is also understood as Jay Pipes points out in [4] that the actual<br> process of provisioning widgets could be time consuming and it is<br> ill-advised to hold a database transaction of any kind open for that<br> duration of time. Ensuring that a user does not exceed some limit on<br> the number of concurrent widgets that he or she may create therefore<br> requires some mechanism to track in-flight requests for widgets. I<br> view these as "intent" but not yet materialized.</span></span></span></p></blockquote><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif">It has nothing to do with the amount of concurrent widgets that a<br> user can create. It's just about the total number of some resource<br> that may be consumed by that user.<br> <br> As for an "intent", I don't believe tracking intent is the right way<br> to go at all. As I've mentioned before, the major problem in Nova's<br> quota system is that there are two tables storing resource usage<br> records: the<br> *actual* resource usage tables (the allocations table in the new<br> resource- providers modeling and the instance_extra, pci_devices and<br> instances table in the legacy modeling) and the *quota usage* tables<br> (quota_usages and reservations tables). The quota_usages table does<br> not need to exist at all, and neither does the reservations table.<br> Don't do intent-based consumption. Instead, just consume (claim) by<br> writing a record for the resource class consumed on a provider into<br> the actual resource usages table and then "check quotas" by querying<br> the *actual* resource usages and comparing the SUM(used) values,<br> grouped by resource class, against the appropriate quota limits for<br> the user. The introduction of the quota_usages and reservations<br> tables to cache usage records is the primary reason for the race<br> problems in the Nova (and<br> other) quota system because every time you introduce a caching system<br> for highly-volatile data (like usage records) you introduce<br> complexity into the write path and the need to track the same thing<br> across multiple writes to different tables needlessly.</span></span></span></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">[amrith] I don't agree, I'll respond to this and the next comment group</span></span></span><br></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">together. See below.</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif">Looking up at this whole infrastructure from the perspective of the<br> database, I think we should require that the database must not be<br> required to operate in any isolation mode higher than<br> READ-COMMITTED; more about that later (i.e. requiring a database run<br> either serializable or repeatable read is a show stopper).</span></span></span></p></blockquote><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif">This is an implementation detail is not relevant to the discussion<br> about what the interface of a quota library would look like.</span></span></span></p></blockquote><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif">[amrith] I disagree, let me give you an example of why.<br> <br> Earlier, I wrote:</span></span></span></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif">Such a hypothetical service may also consume resources from other<br> services that it wishes to track, and impose limits on.</span></span></span></p></blockquote></blockquote><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif">And you responded:</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif">Yes, absolutely agreed.</span></span></span><br></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">So let's take this hypothetical service that in response to a user</span></span></span><br></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">request, will provision a Cinder volume and a Nova instance. Let's assume<br> that the service also imposes limits on the number of cinder volumes and<br> nova instances the user may provision; independent of limits that Nova and<br> Cinder may themselves maintain.</span></span></span></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif">One way that the hypothetical service can function is this:<br> <br> (a) check Cinder quota, if successful, create cinder volume<br> (b) check Nova quota, if successful, create nova instance with cinder<br> volume attachment<br> <br> Now, this is sub-optimal as there are going to be some number of cases</span></span></span></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">where the nova quota check fails. Now you have needlessly created and will<br> have to release a cinder volume. It also takes longer to fail.</span></span></span></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif">Another way to do this is this:<br> <br> (1) check Cinder quota, if successful, check Nova quota, if successful<br> proceed to (2) else error out<br> (2) create cinder volume<br> (3) create nova instance with cinder attachment.<br> <br> I'm trying to get to this latter form of doing things.<br> <br> Easy, you might say ... theoretically this should simply be:<br> <br> BEGIN;<br> -- Get data to do the Cinder check<br> <br> SELECT ......<br> <br> -- Do the cinder check<br> <br> INSERT INTO ....<br> <br> -- Get data to do the Nova check<br> <br> SELECT ....<br> <br> -- Do the Nova check<br> <br> INSERT INTO ...<br> <br> COMMIT<br> <br> You can only make this work if you ran at isolation level serializable.</span></span></span></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">Why?</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif">To make this run at isolation level REPEATABLE-READ, you must enforce</span></span></span><br></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">constraints at the database level that will fail the commit. But wait, you<br> can't do that because the data about the global limits may not be in the<br> same database as the usage records. Later you talk about caching and<br> stuff; all that doesn't help a database constraint.</span></span></span></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif">For this reason, I think there is going to have to be some cognizance to</span></span></span><br></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">the database isolation level in the design of the library, and I think it<br> will also impact the API that can be constructed.</span></span></span></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif">In general therefore, I believe that the hypothetical service<br> processing requests for widgets would have to handle three kinds of<br> operations, provision, modify, and destroy. The names are, I<br> believe, self-explanatory.</span></span></span></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">Generally, modification of a resource doesn't come into play. The<br> primary exception to this is for transferring of ownership of some</span></span></span></p></blockquote></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">resource.</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif">[amrith] Trove RESIZE is a huge benefit for users and while it may be a</span></span></span><br></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">pain as you say, this is still a very real benefit. Trove allows you to<br> resize both your storage (resize the cinder volume) and resize your<br> instance (change the flavor).</span></span></span></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""> </span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif">Without loss of generality, one can say that all three of them must<br> validate that the operation does not violate some limit (no more<br> than X widgets, no fewer than X widgets, rates, and so on).</span></span></span></p></blockquote><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif">No, only the creation (and very rarely the modification) needs any<br> validation that a limit could been violated. Destroying a resource<br> never needs to be checked for limit violations.</span></span></span></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">[amrith] Well, if you are going to create a volume of 10GB and your</span></span></span><br></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">limit is 100GB, resizing it to 200GB should fail, I think.</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif">Assuming that the service provisions resources from other services,<br> it is also conceivable that limits be imposed on the quantum of<br> those services consumed. In practice, I can imagine a service like<br> Trove using the Delimiter project to perform all of these kinds of<br> limit checks; I'm not suggesting that it does this today, nor that<br> there is an immediate plan to implement all of them, just that these<br> all seem like good uses a Quota Management capability.<br> <br> - User may not have more than 25 database instances at a</span></span></span></p></blockquote></blockquote></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">time</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> - User may not have more than 4 clusters at a time<br> - User may not consume more than 3TB of SSD storage at a<br> time</span></span></span></p></blockquote><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif">Only if SSD storage is a distinct resource class from DISK_GB. Right<br> now, Nova makes no differentiation w.r.t. SSD or HDD or shared vs.<br> local block storage.</span></span></span></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">[amrith] It matters not to Trove whether Nova does nor not. Cinder</span></span></span><br></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">supports volume-types and users DO want to limit based on volume-type (for<br> example).</span></span></span></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> - User may not launch more than 10 huge instances at a<br> time</span></span></span></p></blockquote><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif">What is the point of such a limit?</span></span></span><br></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">[amrith] Metering usage, placing limitations on the quantum of resources</span></span></span><br></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">that a user may provision. Same as with Nova. A flavor is merely a simple<br> way to tie together a bag of resources. It is a way to restrict access,<br> for example, to specific resources that are available in the cloud. HUGE<br> is just an example I gave, pick any flavor you want, and here's how a<br> service like Trove uses it.</span></span></span></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif">Users can ask to launch an instance of a specific database+version;</span></span></span><br></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">MySQL 5.6-48 for example. Now, an operator can restrict the instance<br> flavors, or volume types that can be associated with the specific<br> datastore. And the flavor could be used to map to, for example whether the<br> instance is running on bare metal or in a VM and if so with what kind of<br> hardware. That's a useful construct for a service like Trove.</span></span></span></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> - User may not launch more than 3 clusters an hour</span></span></span><br></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">-1. This is rate limiting and should be handled by rate-limiting</span></span></span><br></p></blockquote></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">services.</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> - No more than 500 copies of Oracle may be run at a time</span></span></span><br></p></blockquote><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif">Is "Oracle" a resource class?</span></span></span><br></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">[amrith] As I view it, every project should be free to define its own</span></span></span><br></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">set of resource classes and meter them as it feels fit. So, while Oracle<br> licenses may not, conceivably a lot of things that Nova, Cinder, and the<br> other core projects don't care about, are in fact relevant for a consumer<br> of this library.</span></span></span></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif">While Nova would be the service that limits the number of instances<br> a user can have at a time, the ability for a service to limit this<br> further should not be underestimated.<br> <br> In turn, should Nova and Cinder also use the same Quota Management<br> Library, they may each impose limitations like:<br> <br> - User may not launch more than 20 huge instances at a<br> time</span></span></span></p></blockquote><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif">Not a useful limitation IMHO.</span></span></span><br></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">[amrith] I beg to differ. Again a huge instance is just an example of</span></span></span><br></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">some flavor; and the idea is to allow a project to place its own metrics<br> and meter based on those.</span></span></span></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> - User may not launch more than 3 instances in a minute</span></span></span><br></p></blockquote><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif">-1. This is rate limiting.</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> - User may not consume more than 15TB of SSD at a time<br> - User may not have more than 30 volumes at a time<br> <br> Again, I'm not implying that either Nova or Cinder should provide<br> these capabilities.<br> <br> With this in mind, I believe that the minimal set of operations that<br> Delimiter should provide are:<br> <br> - define_resource(name, max, min, user_max, user_min, ...)</span></span></span></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">What would the above do? What service would it be speaking to?</span></span></span><br></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">[amrith] I assume that this would speak with some backend (either</span></span></span><br></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">keystone or the project itself) and record these designated limits. This<br> is the way to register a project specific metric like "Oracle licenses".</span></span></span></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> - update_resource_limits(name, user, user_max, user_min,<br> ...)</span></span></span></p></blockquote><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif">This doesn't belong in a quota library. It belongs as a REST API in<br> Keystone.</span></span></span></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">[amrith] Fine, same place where the previous thing stores the global</span></span></span><br></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">defaults is the target of this call.</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""> </span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> - reserve_resource(name, user, size, parent_resource, ...)</span></span></span><br></p></blockquote><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif">This doesn't belong in a quota library at all. I think reservations<br> are not germane to resource consumption and should be handled by an<br> external service at the orchestration layer.</span></span></span></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">[amrith] Again not true, as illustrated above this library is the thing</span></span></span><br></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">that projects could use to determine whether or not to honor a request.<br> This reserve/provision process is, I believe required because of the<br> vagaries of how we want to implement this in the database.</span></span></span></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> - provision_resource(resource, id)</span></span></span><br></p></blockquote><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif">A quota library should not be provisioning anything. A quota library<br> should simply provide a consistent interface for *checking* that a<br> structured request for some set of resources *can* be provided by the<br> service.</span></span></span></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">[amrith] This does not actually call Nova or anything; merely that AFTER</span></span></span><br></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">the hypothetical service has called NOVA, this converts the reservation<br> (which can expire) into an actual allocation.</span></span></span></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> - update_resource(id or resource, newsize)</span></span></span><br></p></blockquote><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif">Resizing resources is a bad idea, IMHO. Resources are easier to deal<br> with when they are considered of immutable size and simple (i.e. not<br> complex or nested). I think the problem here is in the definition of<br> resource classes improperly.</span></span></span></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">[amrith] Let's leave the quota library aside. This assertion strikes at</span></span></span><br></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">the very heart of things like Nova resize, or for that matter Cinder<br> volume resize. Are those all bad ideas? I made a 500GB Cinder volume and<br> it is getting close to full. I'd like to resize it to 2TB. Are you saying<br> that's not a valid use case?</span></span></span></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif">For example, a "cluster" is not a resource. It is a collection of<br> resources of type node. "Resizing" a cluster is a misnomer, because<br> you aren't resizing a resource at all. Instead, you are creating or<br> destroying resources inside the cluster (i.e. joining or leaving</span></span></span></p></blockquote></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">cluster nodes).</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif">BTW, this is also why the "resize instance" API in Nova is such a<br> giant pain in the ass. It's attempting to "modify" the instance</span></span></span></p></blockquote></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">"resource"</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif">when the instance isn't really the resource at all. The VCPU, RAM_MB,<br> DISK_GB, and PCI devices are the actual resources. The instance is a<br> convenient way to tie those resources together, and doing a "resize"<br> of the instance behind the scenes actually performs a *move*<br> operation, which isn't a *change* of the original resources. Rather,<br> it is a creation of a new set of resources (of the new amounts) and a<br> deletion of the old set of resources.</span></span></span></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">[amrith] that's fine, if all we want is to handle the resize operation</span></span></span><br></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">as a new instance followed by a deletion, that's great. But that semantic<br> isn't necessarily the case for something like (say) cinder.</span></span></span></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif">The "resize" API call adds some nasty confirmation and cancel<br> semantics to the calling interface that hint that the underlying<br> implementation of the "resize" operation is in actuality not a resize<br> at all, but rather a create-new-and-delete-old-resources operation.</span></span></span></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">[amrith] And that isn't germane to a quota library, I don't think. What</span></span></span><br></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">is, is this. Do we want to treat the transient state when there are (for<br> example of Nova) two instances, one of the new flavor and one of the old<br> flavor, or not. But, from the perspective of a quota library, a resize<br> operation is merely a reset of the quota by the delta in the resource<br> consumed.</span></span></span></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""> </span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> - release_resource(id or resource)<br> - expire_reservations()</span></span></span></p></blockquote><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif">I see no need to have reservations in the quota library at all, as<br> mentioned above.</span></span></span></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">[amrith] Then I think the quota library must require that either (a) the</span></span></span><br></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">underlying database runs serializable or (b) database constraints can be<br> used to enforce that at commit the global limits are adhered to.</span></span></span></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif">As for your proposed interface and calling structure below, I think a<br> much simpler proposal would work better. I'll work on a cross-project<br> spec that describes this simpler proposal, but the basics would be:<br> <br> 1) Have Keystone store quota information for defaults (per service<br> endpoint), for tenants and for users.<br> <br> Keystone would have the set of canonical resource class names, and<br> each project, upon handling a new resource class, would be<br> responsible for a change submitted to Keystone to add the new resource</span></span></span></p></blockquote></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">class code.</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif">Straw man REST API:<br> <br> GET /quotas/resource-classes<br> 200 OK<br> {<br> "resource_classes": {<br> "compute.vcpu": {<br> "service": "compute",<br> "code": "compute.vcpu",<br> "description": "A virtual CPU unit"<br> },<br> "compute.ram_mb": {<br> "service": "compute",<br> "code": "compute.ram_mb",<br> "description": "Memory in megabytes"<br> },<br> ...<br> "volume.disk_gb": {<br> "service": "volume",<br> "code": "volume.disk_gb",<br> "description": "Amount of disk space in gigabytes"<br> },<br> ...<br> "database.count": {<br> "service": "database",<br> "code": "database.count",<br> "description": "Number of database instances"<br> }<br> }<br> }</span></span></span></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">[amrith] Well, a user is allowed to have a certain compute quota (which</span></span></span><br></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">is shared by Nova and Trove) but also a Trove quota. How would your<br> representation represent that?</span></span></span></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif"># Get the default limits for new users...<br> GET /quotas/defaults<br> 200 OK<br> {<br> "quotas": {<br> "compute.vcpu": 100,<br> "compute.ram_mb": 32768,<br> "volume.disk_gb": 1000,<br> "database.count": 25<br> }<br> }<br> <br> # Get a specific user's limits...<br> GET /quotas/users/{UUID}<br> 200 OK<br> {<br> "quotas": {<br> "compute.vcpu": 100,<br> "compute.ram_mb": 32768,<br> "volume.disk_gb": 1000,<br> "database.count": 25<br> }<br> }<br> <br> # Get a tenant's limits...<br> GET /quotas/tenants/{UUID}<br> 200 OK<br> {<br> "quotas": {<br> "compute.vcpu": 1000,<br> "compute.ram_mb": 327680,<br> "volume.disk_gb": 10000,<br> "database.count": 250<br> }<br> }<br> <br> 2) Have Delimiter communicate with the above proposed new Keystone<br> REST API and package up data into an oslo.versioned_objects interface.<br> <br> Clearly all of the above can be heavily cached both on the server and<br> client side since they rarely change but are read often.</span></span></span></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">[amrith] Caching on the client won't save you from oversubscription if</span></span></span><br></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">you don't run serializable.</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif">The Delimiter library could be used to provide a calling interface<br> for service projects to get a user's limits for a set of resource</span></span></span></p></blockquote></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">classes:</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif">(please excuse wrongness, typos, and other stuff below, it's just a<br> straw- man not production working code...)<br> <br> # file: delimiter/objects/limits.py<br> import oslo.versioned_objects.base as ovo import<br> oslo.versioned_objects.fields as ovo_fields<br> <br> <br> class ResourceLimit(ovo.VersionedObjectBase):<br> # 1.0: Initial version<br> VERSION = '1.0'<br> <br> fields = {<br> 'resource_class': ovo_fields.StringField(),<br> 'amount': ovo_fields.IntegerField(),<br> }<br> <br> <br> class ResourceLimitList(ovo.VersionedObjectBase):<br> # 1.0: Initial version<br> VERSION = '1.0'<br> <br> fields = {<br> 'resources': ListOfObjectsField(ResourceLimit),<br> }<br> <br> @cache_this_heavily<br> @remotable_classmethod<br> def get_all_by_user(cls, user_uuid):<br> """Returns a Limits object that tells the caller what a user's<br> absolute limits for the set of resource classes in the system.<br> """<br> # Grab a keystone client session object and connect to Keystone<br> ks = ksclient.Session(...)<br> raw_limits = ksclient.get_limits_by_user()<br> return cls(resources=[ResourceLimit(**d) for d in raw_limits])<br> <br> 3) Each service project would be responsible for handling the<br> consumption of a set of requested resource amounts in an atomic and</span></span></span></p></blockquote></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">consistent way.</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif">[amrith] This is where the rubber meets the road. What is that atomic</span></span></span><br></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">and consistent way? And what computing infrastructure do you need to<br> deliver this?</span></span></span></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif">The Delimiter library would return the limits that the service would<br> pre- check before claiming the resources and either post-check after<br> claim or utilize a compare-and-update technique with a<br> generation/timestamp during claiming to prevent race conditions.<br> <br> For instance, in Nova with the new resource providers database schema<br> and doing claims in the scheduler (a proposed change), we might do<br> something to the effect of:<br> <br> from delimiter import objects as delim_obj from delimier import<br> exceptions as delim_exc from nova import objects as nova_obj<br> <br> request = nova_obj.RequestSpec.get_by_uuid(request_uuid)<br> requested = request.resources<br> limits = delim_obj.ResourceLimitList.get_all_by_user(user_uuid)<br> allocations = nova_obj.AllocationList.get_all_by_user(user_uuid)<br> <br> # Pre-check for violations<br> for resource_class, requested_amount in requested.items():<br> limit_idx = limits.resources.index(resource_class)<br> resource_limit = limits.resources[limit_idx].amount<br> alloc_idx = allocations.resources.index(resource_class)<br> resource_used = allocations.resources[alloc_idx]<br> if (resource_used + requested_amount)> resource_limit:<br> raise delim_exc.QuotaExceeded</span></span></span></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">[amrith] Is the above code run with some global mutex to prevent that</span></span></span><br></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">two people don't believe that they are good on quota at the same time?</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif"># Do claims in scheduler in an atomic, consistent fashion...<br> claims = scheduler_client.claim_resources(request)</span></span></span></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">[amrith] Yes, each 'atomic' claim on a repeatable-read database could</span></span></span><br></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">result in oversubscription.</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif"># Post-check for violations<br> allocations = nova_obj.AllocationList.get_all_by_user(user_uuid)<br> # allocations now include the claimed resources from the scheduler<br> <br> for resource_class, requested_amount in requested.items():<br> limit_idx = limits.resources.index(resource_class)<br> resource_limit = limits.resources[limit_idx].amount<br> alloc_idx = allocations.resources.index(resource_class)<br> resource_used = allocations.resources[alloc_idx]<br> if resource_used> resource_limit:<br> # Delete the allocation records for the resources just claimed<br> delete_resources(claims)<br> raise delim_exc.QuotaExceeded</span></span></span></p></blockquote><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif">[amrith] Again, two people could drive through this code and both of<br> them could fail :(</span></span></span></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif">4) The only other thing that would need to be done for a first go of<br> the Delimiter library is some event listener that can listen for<br> changes to the quota limits for a user/tenant/default in Keystone.<br> We'd want the services to be able notify someone if a reduction in<br> quota results in an overquota situation.<br> <br> Anyway, that's my idea. Keep the Delimiter library small and focused<br> on describing the limits only, not on the resource allocations. Have<br> the Delimiter library present a versioned object interface so the<br> interaction between the data exposed by the Keystone REST API for<br> quotas can evolve naturally and smoothly over time.<br> <br> Best,<br> -jay</span></span></span></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif">Let me illustrate the way I see these things fitting together. A<br> hypothetical Trove system may be setup as follows:<br> <br> - No more than 2000 database instances in total, 300<br> clusters</span></span></span></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">in</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> total<br> - Users may not launch more than 25 database instances, or 4<br> clusters<br> - The particular user 'amrith' is limited to 2 databases and</span></span></span></p></blockquote></blockquote></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">1</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> cluster<br> - No user may consume more than 20TB of storage at a time<br> - No user may consume more than 10GB of memory at a time<br> <br> At startup, I believe that the system would make the following<br> sequence of calls:<br> <br> - define_resource(databaseInstance, 2000, 0, 25, 0, ...)<br> - update_resource_limits(databaseInstance, amrith, 2, 0,</span></span></span></p></blockquote></blockquote></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">...)</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> - define_resource(databaseCluster, 300, 0, 4, 0, ...)<br> - update_resource_limits(databaseCluster, amrith, 1, 0, ...)<br> - define_resource(storage, -1, 0, 20TB, 0, ...)<br> - define_resource(memory, -1, 0, 10GB, 0, ...)<br> <br> Assume that the user john comes along and asks for a cluster with 4<br> nodes, 1TB storage per node and each node having 1GB of memory, the<br> system would go through the following sequence:<br> <br> - reserve_resource(databaseCluster, john, 1, None)<br> o this returns a resourceID (say cluster-resource-</span></span></span></p></blockquote></blockquote></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">ID)</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> o the cluster instance that it reserves counts</span></span></span><br></p></blockquote></blockquote></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">against</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> the limit of 300 cluster instances in total, as well</span></span></span><br></p></blockquote></blockquote></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">as</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> the 4 clusters that john can provision. If 'amrith'</span></span></span><br></p></blockquote></blockquote></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">had</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> requested it, that would have been counted against</span></span></span><br></p></blockquote></blockquote></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">the</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> limit of 2 clusters for the user.<br> <br> - reserve_resource(databaseInstance, john, 1,<br> cluster-resource-id)<br> - reserve_resource(databaseInstance, john, 1,<br> cluster-resource-id)<br> - reserve_resource(databaseInstance, john, 1,<br> cluster-resource-id)<br> - reserve_resource(databaseInstance, john, 1,<br> cluster-resource-id)<br> o this returns four resource id's, let's say<br> instance-1-id, instance-2-id, instance-3-id,<br> instance-4-id<br> o note that each instance is that, an instance by<br> itself. it is therefore not right to consider this</span></span></span></p></blockquote></blockquote></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">as</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> equivalent to a call to reserve_resource() with a</span></span></span><br></p></blockquote></blockquote></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">size</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> of 4, especially because each instance could later</span></span></span><br></p></blockquote></blockquote></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">be</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> tracked as an individual Nova instance.<br> <br> - reserve_resource(storage, john, 1TB, instance-1-id)<br> - reserve_resource(storage, john, 1TB, instance-2-id)<br> - reserve_resource(storage, john, 1TB, instance-3-id)<br> - reserve_resource(storage, john, 1TB, instance-4-id)<br> <br> o each of them returns some resourceID, let's say</span></span></span></p></blockquote></blockquote></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">they</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> returned cinder-1-id, cinder-2-id, cinder-3-id,<br> cinder-4-id<br> o since the storage of 1TB is a unit, it is treated</span></span></span></p></blockquote></blockquote></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">as</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> such. In other words, you don't need to invoke<br> reserve_resource 10^12 times, once per byte<br> allocated<br> :)<br> <br> - reserve_resource(memory, john, 1GB, instance-1-id)<br> - reserve_resource(memory, john, 1GB, instance-2-id)<br> - reserve_resource(memory, john, 1GB, instance-3-id)<br> - reserve_resource(memory, john, 1GB, instance-4-id)<br> o each of these return something, say<br> Dg4KBQcODAENBQEGBAcEDA, CgMJAg8FBQ8GDwgLBA8FAg,<br> BAQJBwYMDwAIAA0DBAkNAg, AQMLDA4OAgEBCQ0MBAMGCA. I</span></span></span></p></blockquote></blockquote></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">have</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> made up arbitrary strings just to highlight that we<br> really don't track these anywhere so we don't care</span></span></span></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">about</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> them.<br> <br> If all this works, then the system knows that John's request does<br> not violate any quotas that it can enforce, it can then go ahead and<br> launch the instances (calling Nova), provision storage, and so on.<br> <br> The system then goes and creates four Cinder volumes, these are<br> cinder-1-uuid, cinder-2-uuid, cinder-3-uuid, cinder-4-uuid.<br> <br> It can then go and confirm those reservations.<br> <br> - provision_resource(cinder-1-id, cinder-1-uuid)<br> - provision_resource(cinder-2-id, cinder-2-uuid)<br> - provision_resource(cinder-3-id, cinder-3-uuid)<br> - provision_resource(cinder-4-id, cinder-4-uuid)<br> <br> It could then go and launch 4 nova instances and similarly provision<br> those resources, and so on. This process could take some minutes and<br> holding a database transaction open for this is the issue that Jay<br> brings up in [4]. We don't have to in this proposed scheme.<br> <br> Since the resources are all hierarchically linked through the<br> overall cluster id, when the cluster is setup, it can finally go and<br> provision<br> that:<br> <br> - provision_resource(cluster-resource-id, cluster-uuid)<br> <br> When Trove is done with some individual resource, it can go and<br> release it. Note that I'm thinking this will invoke release_resource<br> with the ID of the underlying object OR the resource.<br> <br> - release_resource(cinder-4-id), and<br> - release_resource(cinder-4-uuid)<br> <br> are therefore identical and indicate that the 4th 1TB volume is now<br> released. How this will be implemented in Python, kwargs or some<br> other mechanism is, I believe, an implementation detail.<br> <br> Finally, it releases the cluster resource by doing this:<br> <br> - release_resource(cluster-resource-id)<br> <br> This would release the cluster and all dependent resources in a<br> single operation.<br> <br> A user may wish to manage a resource that was provisioned from the<br> service. Assume that this results in a resizing of the instances,<br> then it is a matter of updating that resource.<br> <br> Assume that the third 1TB volume is being resized to 2TB, then it is<br> merely a matter of invoking:<br> <br> - update_resource(cinder-3-uuid, 2TB)<br> <br> Delimiter can go figure out that cinder-3-uuid is a 1TB device and<br> therefore this is an increase of 1TB and verify that this is within<br> the quotas allowed for the user.<br> <br> The thing that I find attractive about this model of maintaining a<br> hierarchy of reservations is that in the event of an error, the<br> service need merely call release_resource() on the highest level<br> reservation and the Delimiter project can walk down the chain and<br> release all the resources or reservations as appropriate.<br> <br> Under the covers I believe that each of these operations should be<br> atomic and may update multiple database tables but these will all be<br> short lived operations.<br> <br> For example, reserving an instance resource would increment the<br> number of instances for the user as well as the number of instances<br> on the whole, and this would be an atomic operation.<br> <br> I have two primary areas of concern about the proposal [3].<br> <br> The first is that it makes the implicit assumption that the<br> "flat mode" is implemented. That provides value to a</span></span></span></p></blockquote></blockquote></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">consumer</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> but I think it leaves a lot for the consumer to do. For</span></span></span><br></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">example,</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> I find it hard to see how the model proposed would handle</span></span></span><br></p></blockquote></blockquote></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">the</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> release of quotas, leave alone the case of a nested<br> release of</span></span></span></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">a</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> hierarchy of resources.<br> <br> The other is the notion that the implementation will begin a<br> transaction, perform a query(), make some manipulations, and<br> then do a save(). This makes for an interesting transaction<br> management challenge as it would require the underlying</span></span></span></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">database</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> to run in an isolation mode of at least repeatable reads and<br> maybe even serializable which would be a performance bear on</span></span></span></p></blockquote></blockquote></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">a</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> heavily loaded system. If run in the traditional read-</span></span></span><br></p></blockquote></blockquote></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">committed</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> mode, this would silently lead to over subscriptions, and</span></span></span><br></p></blockquote></blockquote></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">the</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p><span><span style=""><span class="font" style="font-family:Georgia, serif"> violation of quota limits.<br> <br> I believe that it should be a requirement that the Delimiter library<br> should be able to run against a database that supports, and is<br> configured for READ-COMMITTED, and should not require anything higher.<br> The model proposed above can certainly be implemented with a<br> database running READ-COMMITTED, and I believe that this is also<br> true with the caveat that the operations will be performed through</span></span></span></p></blockquote></blockquote></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">SQLAlchemy.</span></span></span><br></p><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><blockquote style="border-top-style:none;border-right-style:none;border-bottom-style:none;border-top-width:initial;border-right-width:initial;border-bottom-width:initial;border-top-color:initial;border-right-color:initial;border-bottom-color:initial;border-image-source:initial;border-image-slice:initial;border-image-width:initial;border-image-outset:initial;border-image-repeat:initial;border-left-style:solid;border-left-color:rgb(204, 204, 204);border-left-width:1pt;padding-top:0in;padding-right:0in;padding-bottom:0in;padding-left:6pt;margin-left:4.8pt;margin-top:5pt;margin-right:0in;margin-bottom:5pt;"><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif">Thanks,<br> <br> -amrith<br> <br> [1] <a href="http://openstack.markmail.org/thread/tkl2jcyvzgifniux"> http://openstack.markmail.org/thread/tkl2jcyvzgifniux</a><br> [2] <a href="http://openstack.markmail.org/thread/3cr7hoeqjmgyle2j"> http://openstack.markmail.org/thread/3cr7hoeqjmgyle2j</a><br> [3] <a href="https://review.openstack.org/#/c/284454/">https://review.openstack.org/#/c/284454/</a><br> [4] <a href="http://markmail.org/message/7ixvezcsj3uyiro6">http://markmail.org/message/7ixvezcsj3uyiro6</a><br> <br> <br> <br> <br> <br> ____________________________________________________________________<br> __ ____ OpenStack Development Mailing List (not for usage questions)<br> Unsubscribe:<br> <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a></span></span></span></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">_____________________________________________________________________<br> _____ OpenStack Development Mailing List (not for usage questions)<br> Unsubscribe:<br> <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a></span></span></span></p></blockquote><p style="margin-bottom:12pt;"><span><span style=""><span class="font" style="font-family:Georgia, serif">______________________________________________________________________<br> ____ OpenStack Development Mailing List (not for usage questions)<br> Unsubscribe:<br> <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a></span></span></span></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif">__________________________________________________________________________<br> OpenStack Development Mailing List (not for usage questions)<br> Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe"> OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a></span></span></span></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif"><br>__________________________________________________________________________<br> OpenStack Development Mailing List (not for usage questions)<br> Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe"> OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a></span></span></span></p></blockquote><p><span><span style=""><span class="font" style="font-family:Georgia, serif"><br>__________________________________________________________________________<br> OpenStack Development Mailing List (not for usage questions)<br> Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe"> OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a></span></span></span></p></div>
</div>
</blockquote></div>
<p><span><span style=""> </span></span><br></p></div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote><div> </div>
<div><u>__________________________________________________________________________</u><br></div>
<div>OpenStack Development Mailing List (not for usage questions)<br></div>
<div>Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe<br></div>
<div><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br></div>
</blockquote><div> </div>
</body>
</html>