<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 90.0pt 72.0pt 90.0pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:207111723;
mso-list-template-ids:1373907340;}
@list l1
{mso-list-id:1656909075;
mso-list-template-ids:1616945190;}
@list l2
{mso-list-id:1848136622;
mso-list-template-ids:-129223402;}
@list l2:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:72.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="color:#1F497D">Hi Julia,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Apologize we were not able to be there to better represent the use case.
<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">PSB<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt">
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b>From:</b> Julia Kreger <juliaashleykreger@gmail.com> <br>
<b>Sent:</b> Monday, October 1, 2018 11:07 PM<br>
<b>To:</b> OpenStack Development Mailing List (not for usage questions) <openstack-dev@lists.openstack.org><br>
<b>Cc:</b> isaku.yamahata@intel.com; Eyal Lavee <elavee@mellanox.com><br>
<b>Subject:</b> Re: [openstack-dev] [ironic][neutron] SmartNics with Ironic<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">Greetings, Comments in-line.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Thanks,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">-Julia<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Sat, Sep 29, 2018 at 11:27 PM Moshe Levi <<a href="mailto:moshele@mellanox.com">moshele@mellanox.com</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Hi Julia,
<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I don't mind to update the ironic spec [1]. Unfortunately, I wasn't in the PTG but I had a sync meeting with Isuku.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">As I see it there is 2 use-cases:<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:54.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1">
<![if !supportLists]><span style="mso-list:Ignore">1.<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]><span dir="LTR"></span>Running the neutron ovs agent in the smartnic<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:54.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1">
<![if !supportLists]><span style="mso-list:Ignore">2.<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]><span dir="LTR"></span>Running the neutron super ovs agent which manage the ovs running on the smartnic.<o:p></o:p></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">My takeaway from the meeting with neutron is that there would not be a neutron ovs agent running on the smartnic. That the configuration would need to be pushed at all times, which is ultimately better security wise if the tenant NIC is
somehow compromised it reduces the control plane exposure.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">[ML] - <span style="color:#1F497D">Can you elaborate on the security concerns with running the neutron ovs agent on the smart NIC?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">If you compare this to the standard virtualization use case, this is as secure if not more secure.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">The tenant image runs in the bare metal host and receives only a network interface/port.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">The host has no way to access the OS/services/agents running on the smart NIC CPUs, in the same way that a tenant image running in a VM has no way to access the services/agents running in the hypervisor.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">It is in fact event more secure, as they are running in physically disjoint hardware and memory (thus not accessible even through side-channel vulnerabilities such as meltdown/spectre).<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:54.0pt;text-indent:-18.0pt;mso-list:l1 level1 lfo2">
<![if !supportLists]><span style="mso-list:Ignore">1.<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]><span dir="LTR"></span><o:p> </o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">It seem that most of the discussion was around the second use-case.<o:p></o:p></p>
</div>
</div>
</blockquote>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">By the time Ironic and Neutron met together, it seemed like the first use case was no longer under consideration. I may be wrong, but very strong preference existed for the second scenario when we met the next day.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">[ML] – <o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">We are seeing great interest on smart NICs for bare metal use cases to allow to provide services (networking, storage and others) to bare metal servers that were previously only possible for VMs.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Conceptually the smart NIC can be thought of as an isolated hypervisor layer for the bare metal host.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">The first service we are targeting in this spec is aligning the bare metal networking with the standard neutron ovs agent.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">The target is to try to align (as possible) the bare metal implementation to the virtualization use case, up to the point of actually running (as possible) the same agents on the smart NIC (again acting as a
hypervisor for the bare metal host use case).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">This allows to reuse/align the implementation, and naturally scales with the number of bare metal servers, as opposed to running the agents on controller nodes, requiring potentially scaling the controllers to
match the number of bare metal servers.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">It also provides a path to providing more advanced services in the smart NIC in the next steps (not limiting the implementation to be OVSDB protocol specific).<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">This is my understanding on the ironic neutron PTG meeting:<o:p></o:p></p>
<ol start="1" type="1">
<li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l2 level1 lfo3">
Ironic cores don't want to change the deployment interface as proposed in [1].<o:p></o:p></li><li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l2 level1 lfo3">
We should a new network_interface for use case 2. But what about the first use case? Should it be a new network_interface as well?<o:p></o:p></li><li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l2 level1 lfo3">
We should delay the port binding until the baremetal is powered on the ovs is running.
<o:p></o:p></li></ol>
<ol start="3" type="1">
<ol start="1" type="a">
<li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l2 level2 lfo3">
For the first use case I was thinking to change the neutron server to just keep the port binding information in the neutron DB. Then when the neutron ovs agent is a live it will retrieve all the baremeal port , add them to the ovsdb and start the neutron ovs
agent fullsync.<o:p></o:p></li><li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l2 level2 lfo3">
For the second use case the agent is alive so the agent itself can monitor the ovsdb of the baremetal and configure it when it up<o:p></o:p></li></ol>
</ol>
<ol start="4" type="1">
<li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l2 level1 lfo3">
How to notify that neutron agent successfully/unsuccessfully bind the port ? <o:p>
</o:p></li></ol>
<ol start="4" type="1">
<ol start="1" type="a">
<li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l2 level2 lfo4">
In both use-cases we should use neutron-ironic notification to make sure the port binding was done successfully.<o:p></o:p></li></ol>
</ol>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Is my understanding correct?<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><o:p> </o:p></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal">Not quite.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">1) We as in Ironic recognize that there would need to be changes, it is the method as to how that we would prefer to be explicit and have chosen by the interface. The underlying behavior needs to be different, and the new network_interface
should support both cases 1 and 2 because that interface contain needed logic for the conductor to determine the appropriate path forward. We should likely also put some guards in to prevent non-smart interfaces from being used in the same configuration due
to the security issues that creates.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">3) I believe this would be more of a matter of the network_interface knowing that the machine is powered up, and attempting to assert configuration through Neutron to push configuration to the smartnic.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">3a) The consensus is that the information to access the smartnic is hardware configuration metadata and that ironic should be the source of truth for information about that hardware. The discussion was push that as needed into neutron to
help enable the attachment. I proposed just including it in the binding profile as a possibility, since it is transient information.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">3b) As I understood it, this would ultimately be the default operating behavior.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">4) Was not discussed, but something along the path is going to have to check and retry as necessary. That item could be in the network_interface code.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">4a) This doesn't exist yet.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>