<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"\@SimSun";
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><a name="_MailEndCompose"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Some thoughts of “Capability” use cases:<o:p></o:p></span></a></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">1. Nova-compute service can discover host capabilities and tag them into its compute-node resource provider record.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">2. Other services such as ironic and cinder may want to manage its resource pool, and they can tag capabilities themselves to that pool.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">3. Admin/user can register user-defined capabilities to resource providers (i.e. a pool or a host)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">4. Capabilities are managed/grouped/abstracted by namespaces, and scheduler can make decisions based on either cap_names or cap_namespaces<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">5. Placement service DON’T have any specific knowledge of a capability, it only know the its name, namespaces, its relationship to resource providers. They are
used for scheduling, capability management and report.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">6. Placement service need to know where a capability comes from(user-defined, nova-defined, or others), so it can have modification control of capabilities, and
it can list existing capabilities according to types.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">I think the above is the normal use cases, please correct me if there are mistakes or add more items.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">-Yingxin<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><a name="_____replyseparator"></a><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Alex Xu [mailto:soulxu@gmail.com]
<br>
<b>Sent:</b> Monday, July 11, 2016 11:22 PM<br>
<b>To:</b> OpenStack Development Mailing List (not for usage questions) <openstack-dev@lists.openstack.org><br>
<b>Cc:</b> Bhandaru, Malini K <malini.k.bhandaru@intel.com>; Cheng, Yingxin <yingxin.cheng@intel.com>; Jin, Yuntong <yuntong.jin@intel.com>; Tan, Lin <lin.tan@intel.com><br>
<b>Subject:</b> Re: [Nova] [RFC] ResourceProviderTags - Manage Capabilities with ResourceProvider<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">Matt mentioned Aggregate in the scheduler meeting, that is good question and also reminder me I should explain the relationship between Aggregate and ResourceProviderTags.<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">The Aggregate is expected to keep as a tool for group the hosts, and just for group hosts. People used to manage Capabilities with Aggregates, they put the hosts with some kind of Capabilities into same Aggregates, and using the metadata
to identify the Capabilities. But Aggregate with metadata is really not very easy to manage. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Thinking of the case:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Host1 have Capability1<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Host2 have Capability1 and Capability2<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Host3 have Capability2 and Capability3.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">With this case, we needs 3 aggregates for each Capability: agg_cap1, agg_cap2, agg_cap3. Then we needs add hosts to the aggregate as below:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">agg_cap1: host1, host2<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">agg_cap2: host2, host3<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">agg_cap3: host3<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">When there are more capabilities and more hosts which needs to manage, the mapping of hosts and aggregate will be more complicate. And there isn't a easy interface to let user to know the specific host have what kind of capabilities.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">The ResourceProviderTags will be a substitution of Aggregate on manage capabilities. With tags, it won't generate a complex mapping. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">For the same case, we just need to add tags on the ResourceProvider. The interface of tags is pretty easy, check out the api-wg guideline <a href="https://github.com/openstack/api-wg/blob/master/guidelines/tags.rst" target="_blank"><span style="font-size:10.5pt">https://github.com/openstack/api-wg/blob/master/guidelines/tags.rst</span></a>.
And the query parameter made the management easy.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">There are also have some user want to write script to manage the Capabilities. Thinking the aggregate, the script will be very hard due to manage the mapping between aggregates and hosts. The script will be very easy with tags, due to tags
just a set of string.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">2016-07-11 19:08 GMT+08:00 Alex Xu <<a href="mailto:soulxu@gmail.com" target="_blank">soulxu@gmail.com</a>>:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal">This propose is about using ResourceProviderTags as a solution to manage Capabilities (Qualitative) in ResourceProvider.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">The ResourceProviderTags is to describe the capabilities which are defined by OpenStack Service (Compute Service,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Storage Service, Network Service etc.) and by users. The ResourceProvider provide resource exposed by a single<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">compute node, some shared resource pool or an external resource-providing service of some sort. As such,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">ResourceProviderTags is also expected to describe the capabilities of single ResourceProvider or the capabilities of<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">ResourcePool.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">The ResourceProviderTags is similar with ServersTags [0] which is implemented in the Nova. The only difference is<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">that the tags is attached to the ResourceProvider. The API endpoint will be " /ResourceProvider/{uuid}/tags", and it<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">will follow the API-WG guideline about Tags [1].<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">As the Tags are just strings, the meaning of Tag isn't defined by Scheduler. The meaning of Tag is defined by<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">OpenStack services or Users. The ResourceProviderTags will only be used for scheduling with a ResourceProviderTags<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">filter.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">The ResourceProviderTags is very easy to cover the cases of single ResourceProvider, ResourcePool and<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">DynamicResouces. Let see those cases one by one.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">For single ResourceProvider case, just see how Nova report ComputeNode's Capabilities. Firstly, Nova is expected<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">to define a standard way to describe the Capabilities which provided by Hypervisor or Hardware. Then those description<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">of Capabilities can be used across the Openstack deployment. So Nova will define a set of Tags. Those Tags should<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">be included with prefix to indicated that this is coming from Nova. Also the naming rule of prefix can be used to catalog<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">the Capabilities. For example, the capabilities can be defined as:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"> COMPUTE_HW_CAP_CPU_AVX<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> COMPUTE_HW_CAP_CPU_SSE<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> ....<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> COMPUTE_HV_CAP_LIVE_MIGRATION<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> COMPUTE_HV_CAP_LIVE_SNAPSHOT<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> ....<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">( The COMPUTE means this is coming from Nova. HW means this is hardware related Capabilities. HV means this is<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> capabilities of Hypervisor. But the catalog of Capabilities can be discussed separated. This propose focus on the<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> ResourceTags. We also have another idea about not using 'PREFIX' to manage the Tags. We can add attributes to the<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> Tags. Then we have more control on the Tags. This will describe separately in the bottom. )<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Nova will create ResourceProvider for the compute node, and report the quantitative stuff, and report capabilities<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">by adding those defined tags to the ResourceProvider at same time. Then those Capabilities are exposed by Nova<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">automatically.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">The capabilities of ComputeNode can be queried through the API "GET /ResourceProviders/{uuid}/tags".<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">For the ResourcePool case, let us use Shared Storage Pool as example. The different Storage Pool may have<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">different capabilities. Maybe one of Pool are using SSD. For expose that Capability, admin user can do as below:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">1. Define the aggregates<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> $AGG_UUID=`openstack aggregate create r1rck0610`<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">2. Create resource pool for shared storage<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> $RP_UUID=`openstack resource-provider create "/mnt/nfs/row1racks0610/" \<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> --aggregate-uuid=$AGG_UUID`<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">3. Update the capacity of shared storage<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> openstack resource-provider set inventory $RP_UUID \<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> --resource-class=DISK_GB \<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> --total=100000 --reserved=1000 \<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> --min-unit=50 --max-unit=10000 --step-size=10 \<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> --allocation-ratio=1.0<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">4. Add the Capabilities of shared storage<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> openstack resource-provider add tags $RP_UUID --tag STORAGE_WITH_SSD<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">In this case, 'STORAGE_WITH_SSD' is defined by Admin user. This is the same with Quantitative, where there<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">isn't agent to report the Quantitative, neither the Qualitative. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">This is also easy to cover the DynamicResource case. Thinking of Ironic, admin will create ResourcePool for<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">same hardware configuration bare-metal machines. Those machines will have the same set of capabilities. So<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">those capabilities will be added to the ResourcePool as tags, this is pretty same with SharedStoragePool case.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">To expose cloud capabilities to users, there is one more API endpoint 'GET /ResourceProviders/Tags'. User can<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">get all the tags. Then user can know what kind of Capabilities the cloud provides. The query parameter<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">will allow user to filter the Tags by the prefix rules.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">This propose is intended to be a solution of managing Capabilities in the scheduler with ResourceProvider. But yes,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">look at how Nova implement the manage of Capabilities, this is just part of solution. The whole solution still needs needs<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">other propose (like [2]) to describe how to model capabilities inside the compute node and propose (like [3]) to<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">describe how to request capabilities.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Manage Tags with attributes<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">=====================<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">As described above, we add prefix to Tags to mark which service this Tag is coming from and which catalog or<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">namespaces of Capabilities this Tags belongs to. An alternative idea is adding attributes to the Tags. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">We can use one attribute tags to mark the origin of capabilities: Which service this capability comes from, or is it a<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">user-defined tag. Another idea is to add an attribute to control the tag visibility. Some of Tags may not expected to<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">be usable by normal users. The third possible attribute is to mark the namespace of this tag to help group tags.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">For example the prefix "HV" and "HW" can be managed in this way.Then we can query/manage those tags very<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">easily from database by attributes.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Reference<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">========<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> [0] <a href="http://specs.openstack.org/openstack/nova-specs/specs/newton/approved/tag-instances.html" target="_blank">
http://specs.openstack.org/openstack/nova-specs/specs/newton/approved/tag-instances.html</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> [1] <a href="https://github.com/openstack/api-wg/blob/master/guidelines/tags.rst" target="_blank">
https://github.com/openstack/api-wg/blob/master/guidelines/tags.rst</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> [2] <a href="https://review.openstack.org/286520" target="_blank">
https://review.openstack.org/286520</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> [3] <a href="https://review.openstack.org/313784" target="_blank">
https://review.openstack.org/313784</a><o:p></o:p></p>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
</body>
</html>