<div dir="ltr"><div>This seems apropos given something I ran into today with /var/lib/libvirt/qemu/save not having enough capacity to store my suspended instances. If the RAM is freed up (and state saved to disk) then sizing the required size of the qemu partition is pretty big. Based on a rough back-of-napkin calculation you could have up to ( ( max_instances_per_host - 1 ) * physical_memory * memory_overcommit_ratio) disk used by saved instances. For a 128GB machine that's 10T of data. I'd be pretty unhappy about having to add that extra disk to my compute nodes :)<br>
<br>If folks really want to release the resources for a suspended machine perhaps an approach similar to what AWS seems to do would work. By this I mean we would save the instance disk/RAM out to the image store available for future resumption and rescheduling on another machine. <br>
<br></div>-Aaron<br></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Jun 26, 2014 at 12:00 AM, Wangpan <span dir="ltr"><<a href="mailto:hzwangpan@corp.netease.com" target="_blank">hzwangpan@corp.netease.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><u></u>
<div style="LINE-HEIGHT:1.3;BORDER-RIGHT-WIDTH:0px;MARGIN:12px;BORDER-TOP-WIDTH:0px;BORDER-BOTTOM-WIDTH:0px;BORDER-LEFT-WIDTH:0px" marginwidth="0" marginheight="0">
<div>
<div><font face="微软雅黑"><font face="微软雅黑"><font face="微软雅黑"><font color="#000000" face="宋体" size="3">Oh, sorry, I misunderstood your
idea,</font></font></font></font></div>
<div><font face="微软雅黑"><font face="微软雅黑"><font face="宋体">Ok, then the last problem is
how to release the vcpus/mem resources of suspended instances on the
compute nodes?</font></font></font></div>
<div><font face="微软雅黑"></font> </div>
<div><font face="微软雅黑"><a href="tel:2014-06-26%C2%A011" value="+12014062611" target="_blank">2014-06-26 11</a>:57 (UTC+8)</font></div><div class="">
<div><font face="微软雅黑">Wangpan</font></div>
<div><font face="微软雅黑"></font> </div>
<div><font face="微软雅黑">----- Original Message -----</font></div>
<div><font face="微软雅黑">> From: Ricky Saltzer
<<a href="mailto:ricky@cloudera.com" target="_blank">ricky@cloudera.com</a>></font></div>
</div><div><div class="h5"><div><font face="微软雅黑">> To:
"Wangpan"<<a href="mailto:hzwangpan@corp.netease.com" target="_blank">hzwangpan@corp.netease.com</a>><br>>
Sent: <a href="tel:2014-06-25%C2%A003" value="+12014062503" target="_blank">2014-06-25 03</a>:58</font></div>
<div><font face="微软雅黑">> Subject: Re: Re: [Openstack] Why doesn't
suspend release vCPUs/memory?</font></div>
<div><font face="微软雅黑">
<table width="100%">
<tbody>
<tr>
<td width="100%">
<blockquote style="BORDER-LEFT:#000000 2px solid;PADDING-LEFT:5px;PADDING-RIGHT:0px;MARGIN-LEFT:5px;MARGIN-RIGHT:0px">
<div dir="ltr">I don't see how there would be a problem if you still
enforced a hard limit of the number of instances a user can have. The
proposal here is suspending would release <i>only</i> vCPUs and memory,
so the user could still potentially exceed their quota by reaching their
allotted amount of instances. Let's say 5/10 instances for a user
accumulated to 20 vCPUs (4 vCPUs per node), suspending those 5 instances
would release the 20 vCPUs back, but those 5 suspended instances would
still count towards their total number of instances quota.
<br><br><br></div>
<div class="gmail_extra"><br><br>
<div class="gmail_quote">On Mon, Jun 23, 2014 at 11:24 PM, Wangpan <span dir="ltr"><<a href="mailto:hzwangpan@corp.netease.com" target="_blank">hzwangpan@corp.netease.com</a>></span> wrote:<br>
<blockquote style="BORDER-LEFT:#ccc 1px solid;MARGIN:0px 0px 0px 0.8ex;PADDING-LEFT:1ex" class="gmail_quote"><u></u>
<div style="LINE-HEIGHT:1.3;BORDER-RIGHT-WIDTH:0px;MARGIN:12px;BORDER-TOP-WIDTH:0px;BORDER-BOTTOM-WIDTH:0px;BORDER-LEFT-WIDTH:0px" marginwidth="0" marginheight="0">
<div>
<div><font face="微软雅黑"><font face="微软雅黑"><font face="微软雅黑"><font color="#000000" face="宋体" size="3">I think if we do this, a serious security
risk is imported, think this use
case:</font></font></font></font></div>
<div><font face="微软雅黑"><font face="微软雅黑"><font face="宋体">1) an user has
quotas like 10 instances, 20 vcpus, 100G ram and 200G
disks</font></font></font></div>
<div><font face="微软雅黑"><font face="微软雅黑"><font face="宋体">2) he boots 10
instances under his quotas</font></font></font></div>
<div><font face="微软雅黑"><font face="微软雅黑"><font face="宋体">3) he suspends all
this instances</font></font></font></div>
<div><font face="微软雅黑"><font face="微软雅黑"><font face="宋体">4) he repeats step
2&3 day and night</font></font></font></div>
<div><font face="微软雅黑"><font face="微软雅黑"><font face="宋体">5) then the cloud
platform will have no resources to supply
eventually</font></font></font></div>
<div><font face="微软雅黑"></font> </div>
<div><font face="微软雅黑"><a href="tel:2014-06-24%C2%A011" value="+12014062411" target="_blank">2014-06-24 11</a>:17 (UTC+8)</font></div><span><font color="#888888">
<div><font face="微软雅黑">Wangpan</font></div></font></span>
<div>
<div>
<div><font face="微软雅黑"></font> </div>
<div><font face="微软雅黑">----- Original Message -----</font></div>
<div><font face="微软雅黑">> From: Ricky Saltzer <<a href="mailto:ricky@cloudera.com" target="_blank">ricky@cloudera.com</a>></font></div>
<div><font face="微软雅黑">> To: "John Griffith"<<a href="mailto:john.griffith@solidfire.com" target="_blank">john.griffith@solidfire.com</a>><br>>
Sent: <a href="tel:2014-06-24%C2%A001" value="+12014062401" target="_blank">2014-06-24 01</a>:05</font></div>
<div><font face="微软雅黑">> Subject: Re: [Openstack] Why doesn't suspend
release vCPUs/memory?</font></div>
<div><font face="微软雅黑">
<table width="100%">
<tbody>
<tr>
<td width="100%">
<blockquote style="BORDER-LEFT:#000000 2px solid;PADDING-LEFT:5px;PADDING-RIGHT:0px;MARGIN-LEFT:5px;MARGIN-RIGHT:0px">
<div dir="ltr">
<div>That seems to be the case, and I can see where you're
coming from, but if the resources aren't released at the quota
level, then they're effectively being used from a user's point
of view. It would be nice if <i>suspend</i> released resources
after the instance is shutdown, and a <i>resume</i> would
reclaim the resources (provided enough are available). For
instance, if I had 210/210 vCPUs used, and I suspend
<i>instance_a</i> with 1 vCPU, and then launch <i>instance_b
</i>with 1 vCPU...<i>instance_b </i>should successfully
deploy, but resuming <i>instance_a</i> should fail with a
quota exceeded exception.<br><br></div></div>
<div class="gmail_extra"><br><br>
<div class="gmail_quote">On Mon, Jun 23, 2014 at 12:54 PM, John
Griffith <span dir="ltr"><<a href="mailto:john.griffith@solidfire.com" target="_blank">john.griffith@solidfire.com</a>></span>
wrote:<br>
<blockquote style="BORDER-LEFT:#ccc 1px solid;MARGIN:0px 0px 0px 0.8ex;PADDING-LEFT:1ex" class="gmail_quote">
<div dir="ltr">
<div style="FONT-FAMILY:courier new,monospace" class="gmail_default"><br></div>
<div class="gmail_extra">
<div>
<div><br><br>
<div class="gmail_quote">On Mon, Jun 23, 2014 at 10:49 AM,
Ricky Saltzer <span dir="ltr"><<a href="mailto:ricky@cloudera.com" target="_blank">ricky@cloudera.com</a>></span> wrote:<br>
<blockquote style="BORDER-LEFT:#ccc 1px solid;MARGIN:0px 0px 0px 0.8ex;PADDING-LEFT:1ex" class="gmail_quote">
<div dir="ltr">Right, the quotas don't seem to be released.
If I have 210/210 vCPUs used, and I suspend an instance
with 4 vCPUs, I still have 210/210 vCPUs used. <br></div>
<div class="gmail_extra">
<div>
<div><br><br>
<div class="gmail_quote">On Mon, Jun 23, 2014 at 11:38 AM,
John Griffith <span dir="ltr"><<a href="mailto:john.griffith@solidfire.com" target="_blank">john.griffith@solidfire.com</a>></span>
wrote:<br>
<blockquote style="BORDER-LEFT:#ccc 1px solid;MARGIN:0px 0px 0px 0.8ex;PADDING-LEFT:1ex" class="gmail_quote">
<div dir="ltr">
<div class="gmail_extra"><br>
<div class="gmail_quote">On Mon, Jun 23, 2014 at 7:45 AM,
Ricky Saltzer <span dir="ltr"><<a href="mailto:ricky@cloudera.com" target="_blank">ricky@cloudera.com</a>></span>
wrote:<br>
<blockquote style="BORDER-LEFT:#ccc 1px solid;MARGIN:0px 0px 0px 0.8ex;PADDING-LEFT:1ex" class="gmail_quote"><a href="https://ask.openstack.org/en/question/32826/why-doesnt-suspend-release-vcpusmemory/" target="_blank">https://ask.openstack.org/en/question/32826/why-doesnt-suspend-release-vcpusmemory/</a></blockquote>
</div><br>
<div style="FONT-FAMILY:'courier new',monospace">My
understanding was always that the instance is no longer
consuming any resources via the virt layer, so in
essence the resources are in fact freed up on the
Compute Node. Quotas and such however aren't
modified (which seems correct to me). Are you
saying you want to see quota's adjusted here?
</div><br></div></div></blockquote></div><br><br clear="all"><br></div></div>
<div>-- <br>
<div dir="ltr">
<div>Ricky Saltzer</div>
<div><a href="http://www.cloudera.com" target="_blank">http://www.cloudera.com</a><br></div>
<div><br></div></div></div></div></blockquote></div></div></div>
<div style="FONT-FAMILY:'courier new',monospace" class="gmail_default">Yeah, I think that makes sense and is
expected, as a user you're still consuming those "items"
even if they're not active. The alternative would be
(which I think is what you're getting at) to actually deduct
items that are suspended from the tenants quota count.
I guess when I think of it though those resources are
still "reserved" even if they're not in use. I suppose
you could do this and then if on resume the quota isn't
there we don't actually resume... but I think this could be
argued either way.</div>
<div style="FONT-FAMILY:'courier new',monospace" class="gmail_default"><br></div>
<div style="FONT-FAMILY:'courier new',monospace" class="gmail_default">Maybe seperate quotas for active vs
suspended?
</div><br></div></div></blockquote></div><br><br clear="all"><br>-- <br>
<div dir="ltr">
<div>Ricky Saltzer</div>
<div><a href="http://www.cloudera.com" target="_blank">http://www.cloudera.com</a><br></div>
<div><br></div></div></div></blockquote></td></tr></tbody></table></font><font face="Times New Roman"></font></div></div></div></div></div></blockquote></div><br><br clear="all"><br>-- <br>
<div dir="ltr">
<div>Ricky Saltzer</div>
<div><a href="http://www.cloudera.com" target="_blank">http://www.cloudera.com</a><br></div>
<div><br></div></div></div></blockquote></td></tr></tbody></table></font><font face="Times New Roman"></font></div></div></div></div></div><br>_______________________________________________<br>
Mailing list: <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
Post to : <a href="mailto:openstack@lists.openstack.org">openstack@lists.openstack.org</a><br>
Unsubscribe : <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
<br></blockquote></div><br></div>