<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Sun, Apr 6, 2014 at 3:27 AM, Tim Bell <span dir="ltr"><<a href="mailto:Tim.Bell@cern.ch" target="_blank">Tim.Bell@cern.ch</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
There is some work on a reservation system going on around the Climate project. Last I saw, there was a proposal to integrate this function into Nova itself.<br></blockquote><div><br></div><div>Yes. The problem with that is that quotas are cross-project functionality, so this really does belong outside of Nova.<br>
<br></div><div>I'd like to focus primarily on the job of quota management for the Boson effort, however, and not get mired in the topic of resource reservations (time-based reservations). Climate is already handling the latter.<br>
<br></div><div>Boson should provide a simple, but flexible, way of managing quotas for a variety of resources in a transactional way, across many projects. Anything outside of that scope should be saved for either Climate or some future milestones, IMO.<br>
<br></div><div>Best,<br>-jay<br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
It would be interesting to explore the different quota options. At CERN, we're trying to work through some of the scenarios where hypervisors are not 100% used and we want to run some additional opportunistic work on them (a la spot market). The dynamic flavors is also an interesting one too to expand and shrink according to the workload.<br>
<br>
Amongst other things, we need a way to ask a VM to gracefully complete its current work (i.e. give it some warning) rather than just shrinking/killing it without notice.<br>
<span class="HOEnZb"><font color="#888888"><br>
Tim<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
> -----Original Message-----<br>
> From: Cazzolato, Sergio J [mailto:<a href="mailto:sergio.j.cazzolato@intel.com">sergio.j.cazzolato@intel.com</a>]<br>
> Sent: 06 April 2014 00:35<br>
> To: Nathanael Burton; Jay Pipes; Joe Topjian<br>
> Cc: <a href="mailto:openstack-operators@lists.openstack.org">openstack-operators@lists.openstack.org</a><br>
> Subject: Re: [Openstack-operators] Quota Templates<br>
><br>
> Nice proposal, do you think it could be done with something like automatic leasing or reservations?<br>
><br>
> The problem I see is that could be a conflict if a project request resources (included in its quotas) but another one is using those.<br>
><br>
> From: Nathanael Burton [mailto:<a href="mailto:nathanael.i.burton@gmail.com">nathanael.i.burton@gmail.com</a>]<br>
> Sent: Saturday, April 05, 2014 7:14 PM<br>
> To: Jay Pipes; Joe Topjian<br>
> Cc: <a href="mailto:openstack-operators@lists.openstack.org">openstack-operators@lists.openstack.org</a><br>
> Subject: Re: [Openstack-operators] Quota Templates<br>
><br>
><br>
> On Apr 5, 2014 5:42 PM, "Jay Pipes" <<a href="mailto:jaypipes@gmail.com">jaypipes@gmail.com</a>> wrote:<br>
> ><br>
> > On Sat, 2014-04-05 at 14:56 -0600, Joe Topjian wrote:<br>
> > > Hi Jay,<br>
> > ><br>
> > > The only problem with this is that what happens if an instance<br>
> > > type's<br>
> > > underlying resource allocation changes? Quotas and<br>
> > > reservations<br>
> > > generally need to be on the actual resources that are<br>
> > > consumed, not on<br>
> > > an abstract representation of those resources like an instance<br>
> > > type, due<br>
> > > to the fact that the abstract representation can changes over<br>
> > > time.<br>
> > ><br>
> > > My idea about the UI above works nicely with the idea of<br>
> > > changeable<br>
> > > instance types. The UI can do simple real-time calculations<br>
> > > about what a<br>
> > > quota allocation represents in terms of the actual resources<br>
> > > used by an<br>
> > > instance type.<br>
> > ><br>
> > ><br>
> > > Yes, but there is nothing limiting a user to only a set amount of a<br>
> > > certain resource in a certain environment.<br>
> > ><br>
> > > Perhaps I am misunderstanding this discussion, but here is an example<br>
> > > scenario of how I see it:<br>
> > ><br>
> > > I have a small set of compute nodes that are configured in such a way<br>
> > > that make them "special" (SSD, GPU, etc). I can put those compute<br>
> > > nodes in a host aggregate and tie a series of flavors to that host<br>
> > > aggregate.<br>
> > ><br>
> > > But since there is only one quota that the user has, there is nothing<br>
> > > that is preventing a user from hogging a large majority of the those<br>
> > > "special" nodes. For example, if a user has a quota of 500 cpus and<br>
> > > 800gb of memory, and if the "special" nodes collectively only have<br>
> > > 1500 cpus and 2400gb of memory, the user could consume 1/3rd of the<br>
> > > special resources.<br>
> > ><br>
> > > However, if I was able to tie a separate quota to that host aggregate,<br>
> > > I could give the user a quota of 50 cpus and 80gb of memory alongside<br>
> > > his standard quota of 500/800 for the rest of the cloud.<br>
> > ><br>
> > > I greatly apologize if this scenario is *not* what Narayan was talking<br>
> > > about. In addition, this is not a feature that I'm personally in<br>
> > > desperate need for, but would definitely implement it if it was<br>
> > > available -- much more than a quota template.<br>
> ><br>
> > Yes, the above scenario indeed is an interesting use case. Part of the<br>
> > issue is that with the addition of the "extra specs" part of the<br>
> > instance type, the instance type no longer represents just the dedicated<br>
> > resources that the a single instance of that type consumes, but also a<br>
> > free-form set of "capabilities" that the instance has -- GPU, SSD, etc.<br>
> ><br>
> > The trick really is going to be about quantifying what those free-form<br>
> > qualitative "tags" like GPU actually represent when it comes to the<br>
> > consumption of resources in Nova. As it stands now, quota management has<br>
> > no real way of differentiating one CPU unit from another. I imagine that<br>
> > one solution to this would be to attach a "weighting factor" to each<br>
> > free-form instance type extra spec that would effectively multiply the<br>
> > actual resource that is consumed by the instance of that type by the<br>
> > factor, thereby producing a quantifiable value.<br>
> ><br>
> > For example, let's say you had these two instance types:<br>
> ><br>
> > m1.small<br>
> > - 2GB memory<br>
> > - 10GB disk<br>
> > - 1 CPU<br>
> ><br>
> > g1.small<br>
> > - 2GB memory<br>
> > - 10GB disk<br>
> > - 1 CPU<br>
> > - extra_specs:<br>
> > - gpu<br>
> > - ssd<br>
> ><br>
> > An admin might assign the "gpu" extra spec a weighting factor of<br>
> > "cpu:1.5" and the "ssd" extra spec a weighting factor of "disk:10.0".<br>
> > This could be used by the quota management system to effectively give<br>
> > the g1.small instance type the following adjusted resource allocation:<br>
> ><br>
> > - 2GB memory<br>
> > - 1.5 CPU<br>
> > - 100GB disk<br>
> ><br>
> > Make sense?<br>
> ><br>
> > -jay<br>
> ><br>
> ><br>
> ><br>
> ><br>
> > _______________________________________________<br>
> > OpenStack-operators mailing list<br>
> > <a href="mailto:OpenStack-operators@lists.openstack.org">OpenStack-operators@lists.openstack.org</a><br>
> > <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a><br>
> We use the per-project flavors to somewhat manage this use case. There are a set of default flavors that every project can see and<br>
> use and then the more exotic flavors are private and shared with projects as needed. The other addition we did is to add a few<br>
> additional quota resources, for example local-gigabytes to manage type/quantity of local instance storage.<br>
> Would love to see additional work with regards to quota management and enforcement. Particularly within private clouds, the<br>
> current quota system doesn't allow for much dynamic growth. I think enhancements to the quota system to allow for something akin<br>
> to "spot instances" would be really useful. Quotas would enforce how much resources a project is guaranteed, but projects could<br>
> elastically grow beyond their quotas if the rest of the cloud was under utilized because other projects were not using all their<br>
> guaranteed resources. At the point in which a project that had been under-utilizing their quota needs to schedule new resources, if<br>
> the scheduler finds the cloud is full it would "reap" resources in a FIFO manner from any projects which were in excess of their<br>
> guaranteed quota. Thus freeing up enough resources for the incoming request.<br>
> Just some thoughts I've had recently.<br>
> Nate<br>
><br>
> _______________________________________________<br>
> OpenStack-operators mailing list<br>
> <a href="mailto:OpenStack-operators@lists.openstack.org">OpenStack-operators@lists.openstack.org</a><br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a><br>
</div></div></blockquote></div><br></div></div>