<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt">
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><a name="_____replyseparator"></a><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Jay Pipes [mailto:jaypipes@gmail.com]
<br>
<b>Sent:</b> Monday, October 23, 2017 12:20 PM<br>
<b>To:</b> OpenStack Development Mailing List <openstack-dev@lists.openstack.org><br>
<b>Subject:</b> Re: [openstack-dev] [ironic] ironic and traits<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">Writing from my phone... May I ask that before you proceed with any plan that uses traits for state information that we have a hangout or videoconference to discuss this? Unfortunately today and tomorrow I'm not able to do a hangout but
I can do one on Wednesday any time of the day.<o:p></o:p></p>
<div>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">[Mooney, Sean K] on the uefi boot topic I did bring up at the ptg that we wanted to standardizes tratis for “verified boot”
<o:p></o:p></span></i></b></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">that included a trait for uefi secure boot enabled and to indicated a hardware root of trust, e.g. intel boot guard or similar<o:p></o:p></span></i></b></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">we distinctly wanted to be able to tag nova compute hosts with those new traits so we could require that vms that request<o:p></o:p></span></i></b></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">a host with uefi secure boot enabled and a hardware root of trust are scheduled only to those nodes.
<o:p></o:p></span></i></b></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></i></b></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">There are many other examples that effect both vms and bare metal such as, ecc/interleaved memory, cluster on die,
<o:p></o:p></span></i></b></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">l3 cache code and data prioritization, vt-d/vt-c, HPET, Hyper threading, power states … all of these feature may be present on the platform<o:p></o:p></span></i></b></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">but I also need to know if they are turned on. Ruling out state in traits means all of this logic will eventually get pushed to scheduler filters<o:p></o:p></span></i></b></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">which will be suboptimal long term as more state is tracked. Software defined infrastructure may be the future but hardware defined software<o:p></o:p></span></i></b></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">is sadly the present…<o:p></o:p></span></i></b></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></i></b></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">I do however think there should be a sperateion between asking for a host that provides x with a trait and asking for x to be configure via<o:p></o:p></span></i></b></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">A trait. The trait secure_boot_enabled should never result in the feature being enabled It should just find a host with it on. If you want<o:p></o:p></span></i></b></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">To request it to be turned on you would request a host with secure_boot_capable as a trait and have a flavor extra spec or image property to request<o:p></o:p></span></i></b></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Ironic to enabled it. these are two very different request and should not be treated the same.
<o:p></o:p></span></i></b></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal">Lemme know!<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">-jay<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Oct 23, 2017 5:01 AM, "Dmitry Tantsur" <<a href="mailto:dtantsur@redhat.com">dtantsur@redhat.com</a>> wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Hi Jay!<o:p></o:p></p>
</div>
<p class="MsoNormal">I appreciate your comments, but I think you're approaching the problem from purely VM point of view. Things simply don't work the same way in bare metal, at least not if we want to provide the same user experience.<o:p></o:p></p>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Sun, Oct 22, 2017 at 2:25 PM, Jay Pipes <<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>> wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal">Sorry for delay, took a week off before starting a new job. Comments inline.<br>
<br>
On 10/16/2017 12:24 PM, Dmitry Tantsur wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal">Hi all,<br>
<br>
I promised John to dump my thoughts on traits to the ML, so here we go :)<br>
<br>
I see two roles of traits (or kinds of traits) for bare metal:<br>
1. traits that say what the node can do already (e.g. "the node is<br>
doing UEFI boot")<br>
2. traits that say what the node can be *configured* to do (e.g. "the node can<br>
boot in UEFI mode")<o:p></o:p></p>
</blockquote>
<p class="MsoNormal"><br>
There's only one role for traits. #2 above. #1 is state information. Traits are not for state information. Traits are only for communicating capabilities of a resource provider (baremetal node).<o:p></o:p></p>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">These are not different, that's what I'm talking about here. No users care about the difference between "this node was put in UEFI mode by an operator in advance", "this node was put in UEFI mode by an ironic driver on demand" and "this
node is always in UEFI mode, because it's AARCH64 and it does not have BIOS". These situation produce the same result (the node is booted in UEFI mode), and thus it's up to ironic to hide this difference.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">My suggestion with traits is one way to do it, I'm not sure what you suggest though.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal"><br>
For example, let's say we add the following to the os-traits library [1]<br>
<br>
* STORAGE_RAID_0<br>
* STORAGE_RAID_1<br>
* STORAGE_RAID_5<br>
* STORAGE_RAID_6<br>
* STORAGE_RAID_10<br>
<br>
The Ironic administrator would add all RAID-related traits to the baremetal nodes that had the *capability* of supporting that particular RAID setup [2]<br>
<br>
When provisioned, the baremetal node would either have RAID configured in a certain level or not configured at all.
<o:p></o:p></p>
</blockquote>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal"><br>
A very important note: the Placement API and Nova scheduler (or future Ironic scheduler) doesn't care about this. At all. I know it sounds like I'm being callous, but I'm not. Placement and scheduling doesn't care about the state of things. It only cares about
the capabilities of target destinations. That's it.<o:p></o:p></p>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Yes, because VMs always start with a clean state, and hypervisor is there to ensure that. We don't have this luxury in ironic :) E.g. our SNMP driver is not even aware of boot modes (or RAID, or BIOS configuration), which does not mean
that a node using it cannot be in UEFI mode (have a RAID or BIOS pre-configured, etc, etc).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal"><o:p> </o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal">This seems confusing, but it's actually very useful. Say, I have a flavor that<br>
requests UEFI boot via a trait. It will match both the nodes that are already in<br>
UEFI mode, as well as nodes that can be put in UEFI mode.<o:p></o:p></p>
</blockquote>
<p class="MsoNormal"><br>
No :) It will only match nodes that have the UEFI capability. The set of providers that have the ability to be booted via UEFI is *always* a superset of the set of providers that *have been booted via UEFI*. Placement and scheduling decisions only care about
that superset -- the providers with a particular capability.<o:p></o:p></p>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Well, no, it will. Again, you're purely basing on the VM idea, where a VM is always *put* in UEFI mode, no matter how the hypervisor looks like. It is simply not the case for us. You have to care what state the node is, because many drivers
cannot change this state.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal"><o:p> </o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal">This idea goes further with deploy templates (new concept we've been thinking<br>
about). A flavor can request something like CUSTOM_RAID_5, and it will match the<br>
nodes that already have RAID 5, or, more interestingly, the nodes on which we<br>
can build RAID 5 before deployment. The UEFI example above can be treated in a<br>
similar way.<br>
<br>
This ends up with two sources of knowledge about traits in ironic:<br>
1. Operators setting something they know about hardware ("this node is in UEFI<br>
mode"),<br>
2. Ironic drivers reporting something they<br>
2.1. know about hardware ("this node is in UEFI mode" - again)<br>
2.2. can do about hardware ("I can put this node in UEFI mode")<o:p></o:p></p>
</blockquote>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
You're correct that both pieces of information are important. However, only the "can do about hardware" part is relevant to Placement and Nova.<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal">For case #1 we are planning on a new CRUD API to set/unset traits for a node.<o:p></o:p></p>
</blockquote>
<p class="MsoNormal"><br>
I would *strongly* advise against this. Traits are not for state information.<br>
<br>
Instead, consider having a DB (or JSON) schema that lists state information in fields that are explicitly for that state information.<br>
<br>
For example, a schema that looks like this:<br>
<br>
{<br>
"boot": {<br>
"mode": <one of 'bios' or 'uefi'>,<br>
"params": <dict><br>
},<br>
"disk": {<br>
"raid": {<br>
"level": <int>,<br>
"controller": <one of 'sw' or 'hw'>,<br>
"driver": <string>,<br>
"params": <dict><br>
}, ...<br>
},<br>
"network": {<br>
...<br>
}<br>
}<br>
<br>
etc, etc.<br>
<br>
Don't use trait strings to represent state information.<o:p></o:p></p>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I don't see an alternative proposal that will satisfy what we have to solve.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
Best,<br>
-jay<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal">Case #2 is more interesting. We have two options, I think:<br>
<br>
a) Operators still set traits on nodes, drivers are simply validating them. E.g.<br>
an operators sets CUSTOM_RAID_5, and the node's RAID interface checks if it is<br>
possible to do. The downside is obvious - with a lot of deploy templates<br>
available it can be a lot of manual work.<br>
<br>
b) Drivers report the traits, and they get somehow added to the traits provided<br>
by an operator. Technically, there are sub-cases again:<br>
b.1) The new traits API returns a union of operator-provided and<br>
driver-provided traits<br>
b.2) The new traits API returns only operator-provided traits; driver-provided<br>
traits are returned e.g. via a new field (node.driver_traits). Then nova will<br>
have to merge the lists itself.<br>
<br>
My personal favorite is the last option: I'd like a clear distinction between<br>
different "sources" of traits, but I'd also like to reduce manual work for<br>
operators.<br>
<br>
A valid counter-argument is: what if an operator wants to override a<br>
driver-provided trait? E.g. a node can do RAID 5, but I don't want this<br>
particular node to do it for any reason. I'm not sure if it's a valid case, and<br>
what to do about it.<br>
<br>
Let me know what you think.<br>
<br>
Dmitry<o:p></o:p></p>
</blockquote>
<p class="MsoNormal"><br>
[1] <a href="http://git.openstack.org/cgit/openstack/os-traits/tree/" target="_blank">
http://git.openstack.org/cgit/openstack/os-traits/tree/</a><br>
[2] Based on how many attached disks the node had, the presence and abilities of a hardware RAID controller, etc<o:p></o:p></p>
<div>
<div>
<p class="MsoNormal"><br>
<br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><o:p></o:p></p>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><o:p></o:p></p>
</blockquote>
</div>
</div>
</div>
</div>
</body>
</html>