<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Miguel Angel Ajo Pelayo [mailto:majopela@redhat.com]
<br>
<b>Sent:</b> Friday, April 08, 2016 4:17 PM<br>
<b>To:</b> OpenStack Development Mailing List (not for usage questions) <openstack-dev@lists.openstack.org><br>
<b>Subject:</b> [openstack-dev] [neutron] [nova] scheduling bandwidth resources / NIC_BW_KB resource class<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<p class="MsoNormal" style="margin-left:.5in">Hi!, <o:p></o:p></p>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"> In the context of [1] (generic resource pools / scheduling in nova) and [2] (minimum bandwidth guarantees -egress- in neutron), I had a talk a few weeks ago with Jay Pipes, <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"> The idea was leveraging the generic resource pools and scheduling mechanisms defined in [1] to find the right hosts and track the total available bandwidth per host (and per host "physical network"), something
in neutron (still to be defined where) would notify the new API about the total amount of "NIC_BW_KB" available on every host/physnet.<o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">I believe that NIC bandwidth can be taken from Libvirt see [4] and the only piece that is missing is to tell nova the mapping of physnet to network interface
name. (In case of SR-IOV this is already known) <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">I see bandwidth (speed) as one of many capabilities of NIC, therefore I think we should take all of them in the same way in this case libvirt. I was think of
adding a new NIC as new resource to nova. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">[4] - <device><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> <name>net_enp129s0_e4_1d_2d_2d_8c_41</name><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> <path>/sys/devices/pci0000:80/0000:80:01.0/0000:81:00.0/net/enp129s0</path><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> <parent>pci_0000_81_00_0</parent><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> <capability type='net'><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> <interface>enp129s0</interface><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> <address>e4:1d:2d:2d:8c:41</address><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> <link speed='40000' state='up'/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> <feature name='rx'/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> <feature name='tx'/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> <feature name='sg'/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> <feature name='tso'/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> <feature name='gso'/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> <feature name='gro'/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> <feature name='rxvlan'/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> <feature name='txvlan'/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> <feature name='rxhash'/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> <feature name='rdma'/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> <capability type='80203'/><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </capability><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"></device><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"> That part is quite clear to me,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"> From [1] I'm not sure which blueprint introduces the ability to schedule based on the resource allocation/availability itself, ("resource-providers-scheduler" seems more like an optimization to the schedule/DB
interaction, right?)<span style="color:#1F497D"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:#1F497D">My understating is that the resource provider blueprint is just a rough filter of compute nodes before passing them to the scheduler filters. The existing filters here [6] will do the accurate filtering of resources.</span>
<span style="color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">see [5] <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">[5] - http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2016-04-04.log.html#t2016-04-04T16:24:10 <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">[6] -<b> </b><a href="http://docs.openstack.org/developer/nova/filter_scheduler.html"><span style="color:#1F497D;text-decoration:none">http://docs.openstack.org/developer/nova/filter_scheduler.html</span></a><b>
<o:p></o:p></b></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"> And, that brings me to another point: at the moment of filtering hosts, nova I guess, will have the neutron port information, it has to somehow identify if the port is tied to a minimum bandwidth QoS policy.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"> That would require identifying that the port has a "qos_policy_id" attached to it, and then, asking neutron for the specific QoS policy [3], then look out for a minimum bandwidth rule (still to be defined),
and extract the required bandwidth from it.<o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">I am not sure if that is the correct way to do it, but you can create NIC bandwidth filter (or NIC capabilities filter) and in it you can implement the way
to retrieve Qos policy information by using neutron client.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"> That moves, again some of the responsibility to examine and understand external resources to nova.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"> Could it make sense to make that part pluggable via stevedore?, so we would provide something that takes the "resource id" (for a port in this case) and returns the requirements translated to resource classes
(NIC_BW_KB in this case).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">Best regards,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">Miguel Ángel Ajo<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">[1] <a href="http://lists.openstack.org/pipermail/openstack-dev/2016-February/086371.html">
http://lists.openstack.org/pipermail/openstack-dev/2016-February/086371.html</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">[2] <a href="https://bugs.launchpad.net/neutron/+bug/1560963">
https://bugs.launchpad.net/neutron/+bug/1560963</a> <o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">[3] <a href="http://developer.openstack.org/api-ref-networking-v2-ext.html#showPolicy">http://developer.openstack.org/api-ref-networking-v2-ext.html#showPolicy</a><o:p></o:p></p>
</div>
</div>
</div>
</body>
</html>