<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-serif;" dir="ltr">
<p style="margin-top:0;margin-bottom:0"></p>
<p style="margin: 0px; font: 12px "Helvetica Neue"; color: rgb(69, 69, 69);">>> that is incorrect both a and by will be returned. the spec states that for host A we report an inventory of 4 VCPUs and</p>
<p style="margin: 0px; font: 12px "Helvetica Neue"; color: rgb(69, 69, 69);">>> an inventory of 4 PCPUs and host B will have 1 inventory of 4 PCPUs so both host will be returned assuming</p>
<p style="margin: 0px; font: 12px "Helvetica Neue"; color: rgb(69, 69, 69);">>> $<no. of cpus> <=4</p>
<p style="margin: 0px; font: 12px "Helvetica Neue"; color: rgb(69, 69, 69); min-height: 14px;">
<br>
</p>
<p style="margin: 0px; font: 12px "Helvetica Neue"; color: rgb(69, 69, 69);">Means if ``vcpu_pin_set`` is set in previous release then report both VCPU and PCPU as inventory (in Train) but this seems contradictory for example:</p>
<p style="margin: 0px; font: 12px "Helvetica Neue"; color: rgb(69, 69, 69); min-height: 14px;">
<br>
</p>
<p style="margin: 0px; font: 12px "Helvetica Neue"; color: rgb(69, 69, 69);">On Stein,<span class="Apple-converted-space"> </span></p>
<p style="margin: 0px; font: 12px "Helvetica Neue"; color: rgb(69, 69, 69); min-height: 14px;">
<br>
</p>
<p style="margin: 0px; font: 12px "Helvetica Neue"; color: rgb(69, 69, 69);">Configuration on compute node A:</p>
<p style="margin: 0px; font: 12px "Helvetica Neue"; color: rgb(69, 69, 69);">vcpu_pin_set=0-3 (This will report 4 VCPUs inventory in placement database)</p>
<p style="margin: 0px; font: 12px "Helvetica Neue"; color: rgb(69, 69, 69); min-height: 14px;">
<br>
</p>
<p style="margin: 0px; font: 12px "Helvetica Neue"; color: rgb(69, 69, 69);">On Train:</p>
<p style="margin: 0px; font: 12px "Helvetica Neue"; color: rgb(69, 69, 69);">vcpu_pin_set=0-3</p>
<p style="margin: 0px; font: 12px "Helvetica Neue"; color: rgb(69, 69, 69); min-height: 14px;">
<br>
</p>
<p style="margin: 0px; font: 12px "Helvetica Neue"; color: rgb(69, 69, 69);">The inventory will be reported as 4 VCPUs and 4 PCPUs in the placement db</p>
<p style="margin: 0px; font: 12px "Helvetica Neue"; color: rgb(69, 69, 69); min-height: 14px;">
<br>
</p>
<p style="margin: 0px; font: 12px "Helvetica Neue"; color: rgb(69, 69, 69);">Now say user wants to create instances as below:</p>
<ol style="list-style-type: decimal;">
<li style="margin: 0px; font: 12px "Helvetica Neue"; color: rgb(69, 69, 69);">Flavor having extra specs (resources:PCPU=1), instance A</li><li style="margin: 0px; font: 12px "Helvetica Neue"; color: rgb(69, 69, 69);">Flavor having extra specs (resources:VCPU=1), instance B</li></ol>
<p style="margin: 0px; font: 12px "Helvetica Neue"; color: rgb(69, 69, 69); min-height: 14px;">
<br>
</p>
<p style="margin: 0px; font: 12px "Helvetica Neue"; color: rgb(69, 69, 69);">For both instance requests, placement will return compute Node A.</p>
<p style="margin: 0px; font: 12px "Helvetica Neue"; color: rgb(69, 69, 69);">Instance A:<span class="Apple-converted-space">
</span>will be pinned to say 0 CPU</p>
<p style="margin: 0px; font: 12px "Helvetica Neue"; color: rgb(69, 69, 69);">Instance B:<span class="Apple-converted-space">
</span>will float on 0-3</p>
<p style="margin: 0px; font: 12px "Helvetica Neue"; color: rgb(69, 69, 69); min-height: 14px;">
<br>
</p>
<p style="margin: 0px; font: 12px "Helvetica Neue"; color: rgb(69, 69, 69);">To resolve above issue, I think it’s possible to detect whether the compute node was configured to be used for pinned instances if ``NumaTopology`` ``pinned_cpus`` attribute is not
empty. In that case, vcpu_pin_set will be reported as PCPU otherwise VCPU.</p>
<p style="margin: 0px; font: 12px "Helvetica Neue"; color: rgb(69, 69, 69);"><br>
</p>
<p style="margin: 0px; font: 12px "Helvetica Neue"; color: rgb(69, 69, 69);">Regards,</p>
<p style="margin: 0px; font: 12px "Helvetica Neue"; color: rgb(69, 69, 69);">-Bhagyashri Shewale-</p>
<p></p>
</div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Sean Mooney <smooney@redhat.com><br>
<b>Sent:</b> Thursday, June 13, 2019 8:32:02 PM<br>
<b>To:</b> Shewale, Bhagyashri; openstack-discuss@lists.openstack.org; openstack@fried.cc; sfinucan@redhat.com; jaypipes@gmail.com<br>
<b>Subject:</b> Re: [nova] Spec: Standardize CPU resource tracking</font>
<div> </div>
</div>
<div class="BodyFragment"><font size="2"><span style="font-size:11pt;">
<div class="PlainText">On Wed, 2019-06-12 at 09:10 +0000, Shewale, Bhagyashri wrote:<br>
> Hi All,<br>
> <br>
> <br>
> Currently I am working on implementation of cpu pinning upgrade part as mentioned in the spec [1].<br>
> <br>
> <br>
> While implementing the scheduler pre-filter as mentioned in [1], I have encountered one big issue:<br>
> <br>
> <br>
> Proposed change in spec: In scheduler pre-filter we are going to alias request_spec.flavor.extra_spec and<br>
> request_spec.image.properties form ``hw:cpu_policy`` to ``resources=(V|P)CPU:${flavor.vcpus}`` of existing instances.<br>
> <br>
> <br>
> So when user will create a new instance or execute instance actions like shelve, unshelve, resize, evacuate and<br>
> migration post upgrade it will go through scheduler pre-filter which will set alias for `hw:cpu_policy` in<br>
> request_spec flavor ``extra specs`` and image metadata properties. In below particular case, it won’t work:-<br>
> <br>
> <br>
> For example:<br>
> <br>
> <br>
> I have two compute nodes say A and B:<br>
> <br>
> <br>
> On Stein:<br>
> <br>
> <br>
> Compute node A configurations:<br>
> <br>
> vcpu_pin_set=0-3 (used as dedicated CPU, This host is added in aggregate which has “pinned” metadata)<br>
vcpu_pin_set does not mean that the host was used for pinned instances<br>
<a href="https://that.guru/blog/cpu-resources/">https://that.guru/blog/cpu-resources/</a><br>
> <br>
> <br>
> Compute node B Configuration:<br>
> <br>
> vcpu_pin_set=0-3 (used as dedicated CPU, This host is added in aggregate which has “pinned” metadata)<br>
> <br>
> <br>
> On Train, two possible scenarios:<br>
> <br>
> Compute node A configurations: (Consider the new cpu pinning implementation is merged into Train)<br>
> <br>
> vcpu_pin_set=0-3 (Keep same settings as in Stein)<br>
> <br>
> <br>
> Compute node B Configuration: (Consider the new cpu pinning implementation is merged into Train)<br>
> <br>
> cpu_dedicated_set=0-3 (change to the new config option)<br>
> <br>
> 1. Consider that one instance say `test ` is created using flavor having old extra specs (hw:cpu_policy=dedicated,<br>
> "aggregate_instance_extra_specs:pinned": "true") in Stein release and now upgraded Nova to Train with the above<br>
> configuration.<br>
> 2. Now when user will perform instance action say shelve/unshelve scheduler pre-filter will change the<br>
> request_spec flavor extra spec from ``hw:cpu_policy`` to ``resources=PCPU:$<no. of cpus>``<br>
it wont remove hw:cpu_policy it will just change the resouces=VCPU:$<no. of cpus> -> resources=PCPU:$<no. of cpus><br>
<br>
> which ultimately will return only compute node B from placement service.<br>
that is incorrect both a and by will be returned. the spec states that for host A we report an inventory of 4 VCPUs and<br>
an inventory of 4 PCPUs and host B will have 1 inventory of 4 PCPUs so both host will be returned assuming<br>
$<no. of cpus> <=4<br>
<br>
> Here, we expect it should have retuned both Compute A and Compute B.<br>
it will<br>
> 3. If user creates a new instance using old extra specs (hw:cpu_policy=dedicated,<br>
> "aggregate_instance_extra_specs:pinned": "true") on Train release with the above configuration then it will return<br>
> only compute node B from placement service where as it should have returned both compute Node A and B.<br>
that is what would have happend in the stien version of the spec and we changed the spec specifically to ensure that<br>
that wont happen. in the train version of the spec you will get both host as candates to prevent this upgrade impact.<br>
> <br>
> Problem: As Compute node A is still configured to be used to boot instances with dedicated CPUs same behavior as<br>
> Stein, it will not be returned by placement service due to the changes in the scheduler pre-filter logic.<br>
> <br>
> <br>
> Propose changes:<br>
> <br>
> <br>
> Earlier in the spec [2]: The online data migration was proposed to change flavor extra specs and image metadata<br>
> properties of request_spec and instance object. Based on the instance host, we can get the NumaTopology of the host<br>
> which will contain the new configuration options set on the compute host. Based on the NumaTopology of host, we can<br>
> change instance and request_spec flavor extra specs.<br>
> <br>
> 1. Remove cpu_policy from extra specs<br>
> 2. Add “resources:PCPU=<count>” in extra specs<br>
> <br>
> <br>
> We can also change the flavor extra specs and image metadata properties of instance and request_spec object using the<br>
> reshape functionality.<br>
> <br>
> <br>
> Please give us your feedback on the proposed solution so that we can update specs accordingly.<br>
i am fairly stongly opposed to useing an online data migration to modify the request spec to reflect the host they<br>
landed on. this speficic problem is why the spec was changed in the train cycle to report dual inventoryis of VCPU and<br>
PCPU if vcpu_pin_set is the only option set or of no options are set.<br>
> <br>
> <br>
> [1]: <a href="https://review.opendev.org/#/c/555081/28/specs/train/approved/cpu-resources.rst@451">
https://review.opendev.org/#/c/555081/28/specs/train/approved/cpu-resources.rst@451</a><br>
> <br>
> [2]: <a href="https://review.opendev.org/#/c/555081/23..28/specs/train/approved/cpu-resources.rst">
https://review.opendev.org/#/c/555081/23..28/specs/train/approved/cpu-resources.rst</a><br>
> <br>
> <br>
> Thanks and Regards,<br>
> <br>
> -Bhagyashri Shewale-<br>
> <br>
> Disclaimer: This email and any attachments are sent in strictest confidence for the sole use of the addressee and may<br>
> contain legally privileged, confidential, and proprietary data. If you are not the intended recipient, please advise<br>
> the sender by replying promptly to this email and then delete and destroy this email and any attachments without any<br>
> further use, copying or forwarding.<br>
<br>
</div>
</span></font></div>
Disclaimer: This email and any attachments are sent in strictest confidence for the sole use of the addressee and may contain legally privileged, confidential, and proprietary data. If you are not the intended recipient, please advise the sender by replying
promptly to this email and then delete and destroy this email and any attachments without any further use, copying or forwarding.
</body>
</html>