<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif; ">
<div>Hi Irena,</div>
<div> Thanks for pointing out an alternative to the network xml solution to live migration. I am still not clear about the solution. </div>
<div><br>
</div>
<div>Some questions:</div>
<ol>
<li>Where does the rename of the PCI device network interface name occur?</li><li>Can this rename be done for a VF? I think your example shows rename of a PF.</li></ol>
<div>Thanks,</div>
<div>Sandhya</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div style="font-family:Calibri; font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style="font-weight:bold">From: </span>Irena Berezovsky <<a href="mailto:irenab@mellanox.com">irenab@mellanox.com</a>><br>
<span style="font-weight:bold">Reply-To: </span>"OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>><br>
<span style="font-weight:bold">Date: </span>Thursday, January 16, 2014 4:43 AM<br>
<span style="font-weight:bold">To: </span>"OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [openstack-dev] [nova] [neutron] PCI pass-through network support<br>
</div>
<div><br>
</div>
<div xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:"Segoe UI";
panose-1:2 11 5 2 4 2 4 2 2 3;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:"Courier New";}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Ian,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Thank you for putting in writing the ongoing discussed specification.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">I have added few comments on the Google doc [1].<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">As for live migration support, this can be done also without libvirt network usage.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Not very elegant, but working: rename the interface of the PCI device to some logical name, let’s say based on neutron port UUID and put it into
the interface XML, i.e.:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">If PCI device network interface name is eth8 and neutron port UUID is 02bc4aec-b4f4-436f-b651-024 then rename it to something like: eth02bc4aec-b4'.
The interface XML will look like this:<o:p></o:p></span></p>
<p style="margin:0in;margin-bottom:.0001pt;line-height:14.25pt;vertical-align:baseline">
<span style="font-size: 10pt; font-family: 'Courier New'; color: black; border: 1pt none windowtext; padding: 0in; "> ...</span><span style="font-size: 10pt; font-family: Helvetica, sans-serif; color: black; "><o:p></o:p></span></p>
<p class="MsoNormal" style="orphans: auto;widows: auto;-webkit-text-stroke-width: 0px;word-spacing:0px">
<span style="font-size: 10pt; font-family: 'Segoe UI', sans-serif; color: black; "><interface type='direct'><br>
<mac address='fa:16:3e:46:d3:e8'/><br>
<source dev='eth02bc4aec-b4' mode='passthrough'/><br>
<target dev='macvtap0'/><br>
<model type='virtio'/><br>
<alias name='net0'/><br>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/><br>
</interface></span><span style="font-size: 10pt; font-family: 'Segoe UI', sans-serif; color: black; "><br>
<br>
</span><o:p></o:p></p>
<p style="margin:0in;margin-bottom:.0001pt;line-height:14.25pt;vertical-align:baseline">
<span style="font-size: 10pt; font-family: 'Courier New'; color: black; border: 1pt none windowtext; padding: 0in; ">...</span><span style="font-size: 10pt; font-family: Helvetica, sans-serif; color: rgb(61, 61, 61); "><o:p></o:p></span></p>
<p class="MsoNormal" style="orphans: auto;widows: auto;-webkit-text-stroke-width: 0px;word-spacing:0px">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">[1]
</span><a href="https://docs.google.com/document/d/1vadqmurlnlvZ5bv3BlUbFeXRS_wh-dsgi5plSjimWjU/edit?pli=1#heading=h.308b0wqn1zde">https://docs.google.com/document/d/1vadqmurlnlvZ5bv3BlUbFeXRS_wh-dsgi5plSjimWjU/edit?pli=1#heading=h.308b0wqn1zde</a><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">BR,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Irena<o:p></o:p></span></p>
<p class="MsoNormal"><b><span style="font-size: 10pt; font-family: Tahoma, sans-serif; ">From:</span></b><span style="font-size: 10pt; font-family: Tahoma, sans-serif; "> Ian Wells [<a href="mailto:ijw.ubuntu@cack.org.uk">mailto:ijw.ubuntu@cack.org.uk</a>]
<br>
<b>Sent:</b> Thursday, January 16, 2014 2:34 AM<br>
<b>To:</b> OpenStack Development Mailing List (not for usage questions)<br>
<b>Subject:</b> Re: [openstack-dev] [nova] [neutron] PCI pass-through network support<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">To clarify a couple of Robert's points, since we had a conversation earlier:<br>
On 15 January 2014 23:47, Robert Li (baoli) <<a href="mailto:baoli@cisco.com" target="_blank">baoli@cisco.com</a>> wrote:<o:p></o:p></p>
<div>
<div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; "> --- do we agree that BDF address (or device id, whatever you call it), and node id shouldn't be used as attributes in defining a PCI flavor?
<o:p></o:p></span></p>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Note that the current spec doesn't actually exclude it as an option. It's just an unwise thing to do. In theory, you could elect to define your flavors using the BDF attribute but determining 'the card in
this slot is equivalent to all the other cards in the same slot in other machines' is probably not the best idea... We could lock it out as an option or we could just assume that administrators wouldn't be daft enough to try.<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; "> * the compute node needs to know the PCI flavor. [...]
<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size: 10.5pt; font-family: Calibri, sans-serif; "> - to support live migration, we need to use it to create network xml<o:p></o:p></span></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I didn't understand this at first and it took me a while to get what Robert meant here.<br>
<br>
This is based on Robert's current code for macvtap based live migration. The issue is that if you wish to migrate a VM and it's tied to a physical interface, you can't guarantee that the same physical interface is going to be used on the target machine, but
at the same time you can't change the libvirt.xml as it comes over with the migrating machine. The answer is to define a network and refer out to it from libvirt.xml. In Robert's current code he's using the group name of the PCI devices to create a network
containing the list of equivalent devices (those in the group) that can be macvtapped. Thus when the host migrates it will find another, equivalent, interface. This falls over in the use case under consideration where a device can be mapped using more than
one flavor, so we have to discard the use case or rethink the implementation.<br>
<br>
There's a more complex solution - I think - where we create a temporary network for each macvtap interface a machine's going to use, with a name based on the instance UUID and port number, and containing the device to map. Before starting the migration we
would create a replacement network containing only the new device on the target host; migration would find the network from the name in the libvirt.xml, and the content of that network would behave identically. We'd be creating libvirt networks on the fly
and a lot more of them, and we'd need decent cleanup code too ('when freeing a PCI device, delete any network it's a member of'), so it all becomes a lot more hairy.<br>
-- <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Ian.<o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</span>
</body>
</html>