Hi Irena, <br><br><div class="gmail_quote">On Sun, Jul 22, 2012 at 5:25 AM, Irena Berezovsky <span dir="ltr"><<a href="mailto:irenab@mellanox.com" target="_blank">irenab@mellanox.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple">
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Hi Young, Gary, Dan,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">I am not sure that my concerns are directly related to the thread subject, but they are related to the thoughts raised by you.<u></u><u></u></span></p>
<p><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><span>1.<span style="font:7.0pt "Times New Roman"">
</span></span></span><u></u><span dir="LTR"></span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Providing compute_host data as part of the 'create_port' can help the Plugin if it requires VM physical location knowledge.
On the other hand, Plugin can 'discover' VM compute_host data by querying OS API.</span></p></div></div></blockquote><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple"><div><p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u><u></u></span></p>
<p><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><span>2.<span style="font:7.0pt "Times New Roman"">
</span></span></span><u></u><span dir="LTR"></span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">What is the live migration flow that currently implemented between Nova and Quantum? As far as I understand nova does not call
Quantum at all. The only work is done by Agents. Agent at the destination host will connect VM tap device and Agent at the source host will disconnect VM tap device. Adding call to Quantum for compute_host data update can be very helpful. Otherwise Agent
notification can be used for getting new Host info update.</span></p></div></div></blockquote><div><br></div><div>Yes, you are correct. Currently there is no webservice API for indicating the hypervisor that will be hosting the workload, and instead plugin itself is responsible for learning about these locations (note: this sometimes takes the form of an agent, but could just be an OpenFlow controller with no local agent). </div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div lang="EN-US" link="blue" vlink="purple"><div><p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u><u></u></span></p>
<p><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><span>3.<span style="font:7.0pt "Times New Roman"">
</span></span></span><u></u><span dir="LTR"></span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">If VM vNIC should be provisioned using SR-IOV technology, the exact Virtual Function data should be put into DOMXML; so it
can be only VIF Driver and not Agent. Since some sort of VF resource management should be implemented per Host, maybe this means that Vif Driver should call Agent for VF allocation. But I think it was decided to avoid such sort of communication.</span></p>
</div></div></blockquote><div>I believe the Cisco vif-plugging logic makes (or at least used to make) a webservice API to an API extension to grab some info required for the libvirt XML. I think that's OK. My main goal is that the vif-plugging logic be simple, so that we don't end up shoving a bunch of network complexity in nova. In the example you're mentioning, is the libvirt interface xml type=direct? If so, perhaps we want to create a standard interface driver for type=direct that makes a generic webservice call to fetch the required parameters. </div>
<div><br></div><div>Dan</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div lang="EN-US" link="blue" vlink="purple"><div><p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Thanks a lot,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Irena<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Dan Wendlandt [mailto:<a href="mailto:dan@nicira.com" target="_blank">dan@nicira.com</a>]
<br>
<b>Sent:</b> Friday, July 20, 2012 12:23 PM<br>
<b>To:</b> <a href="mailto:gkotton@redhat.com" target="_blank">gkotton@redhat.com</a><br>
<b>Cc:</b> OpenStack Development Mailing List<br>
<b>Subject:</b> [openstack-dev] [Quantum] Re: Some thoughts about scalable agent and notification<u></u><u></u></span></p><div><div>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Hi Yong, Gary, <u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Please put [Quantum] in the subject line for such emails, so it is easier for team members to filter. I've edited the subject in this case. More thoughts inline. <u></u><u></u></p>
<div>
<p class="MsoNormal">On Fri, Jul 20, 2012 at 12:30 AM, Gary Kotton <<a href="mailto:gkotton@redhat.com" target="_blank">gkotton@redhat.com</a>> wrote:<u></u><u></u></p>
<div>
<p class="MsoNormal">Hi,<br>
Please see my comments below.<br>
Thanks<br>
Gary<u></u><u></u></p>
<div>
<p class="MsoNormal"><br>
<br>
On 07/20/2012 03:07 AM, Yong Sheng Gong wrote: <u></u><u></u></p>
<p class="MsoNormal"><span style="font-family:"Verdana","sans-serif""><br>
hi,<br>
I think the workflow can be this:<br>
1. nova quantum api: allocate_for_instance(compute_host, device_id)<br>
2.quantum-server: create_port(compute_host, device_id), we need to extend port model to include compute_host</span><u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal">I think that this is problematic in a number of respects:<br>
1. not sure how this will work for live migration (that is moving a VM that is running on host1 to host2)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Nova could probably update the port to point to a new host on live migration. <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<blockquote style="border:none;border-right:solid #cccccc 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"> 2. I am not sure it would be wise to move this information to Quantum. What you have escibed may work for Quantum in OpenStack but Quantum in oVirt may behave differently. The API should be as generic as
possible.<u></u><u></u></p>
</div>
</blockquote>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">I agree that we want to be careful about this. Quantum should be designed such that it can work well with Nova, but I don't think the core API should be nova-specific. The core Quantum API will also be used to connect other OpenStack
services that need to plug into networks (e.g., load-balancer as a service...) as well as other compute stacks all together (e.g., oVirt, as mentioned by garyk). <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<blockquote style="border:none;border-right:solid #cccccc 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal"><br>
<br>
<u></u><u></u></p>
<p class="MsoNormal"><span style="font-family:"Verdana","sans-serif"">4. plugin agent on compute_node: create tap device and attach it to bridge, set vlan and so on, return</span><u></u><u></u></p>
</div>
</div>
</blockquote>
<div>
<p>I'm worried that this is not sufficiently generic. In several cases, it is the compute platform that needs to create the device that represents the vNIC. My guess is that this model that you describe would primarily work for libvirt type=ethernet, I believe,
and that model has a several limitations. Other approaches that are better integrated with libvirt have libvirt create and attach the device based on libvirt XML (checkout out libvirt <interface> elements that have type=bridge or type=direct). There are
also vif-drivers for other platforms like XenServer that definitely don't go create tap devices. <u></u><u></u></p>
<p>I don't think this is sufficiently generic. In several cases, it is the compute platform that needs to create the device that represents the vNIC. My guess is that this model that you describe would primarily work for libvirt type=ethernet, I believe,
and that model has a several limitations. Other approaches that are better integrated with libvirt have libvirt create and attach the device based on libvirt XML (checkout out libvirt <interface> elements that have type=bridge or type=direct). There are
also vif-drivers for other platforms like XenServer that definitely don't go create tap devices. <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<blockquote style="border:none;border-right:solid #cccccc 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"><span style="font-family:"Verdana","sans-serif"">5. quantum -server return the network information to nova, and then nova create VM.<br>
<br>
This workflow differentiates at:<br>
1. tap device is not created by nova, it is created by plugin agent. It will help us to use one unified vif-driver on the nova side for quantum.</span><u></u><u></u></p>
</blockquote>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p>For the same reasons I mentioned above, I believe the complexity of several vif-drivers in nova, while undesirable, is actually difficult to avoid. <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<blockquote style="border:none;border-right:solid #cccccc 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"><span style="font-family:"Verdana","sans-serif""><br>
For notification feature, I hope keep it for metering's purpose.</span><u></u><u></u></p>
</blockquote>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal">I do not think that we should mix the features. The metering is a feature that I think is used for billing. They may use the same infrastructure but I do not think that we may need different approaches for both.<u></u><u></u></p>
</div>
</blockquote>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">There are many things that will need to trigger off basic Quantum events (port creation/update/delete, floating ip allocation, etc.). Even though there will ultimately be different type of consumers (plugin agents, services like DHCP/DNS,
metering, troubleshooting/logging, etc.) I'm hoping we can build a solid base notification mechanism that can be leveraged by many/all of them. If there are conflicting goals for different, perhaps we cannot, but I think we should first discuss what those
conflicting goals, as they will inform our technical design. <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Thanks,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Dan<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<blockquote style="border:none;border-right:solid #cccccc 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-family:"Verdana","sans-serif""><br>
Thanks<br>
Yong Sheng Gong<br>
<br>
</span><u></u><u></u></p>
</blockquote>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</blockquote>
</div>
<p class="MsoNormal"><br>
<br clear="all">
<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal">-- <br>
~~~~~~~~~~~~~~~~~~~~~~~~~~~<br>
Dan Wendlandt <u></u><u></u></p>
<div>
<p class="MsoNormal">Nicira, Inc: <a href="http://www.nicira.com" target="_blank">www.nicira.com</a><u></u><u></u></p>
<div>
<p class="MsoNormal">twitter: danwendlandt<br>
~~~~~~~~~~~~~~~~~~~~~~~~~~~<u></u><u></u></p>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div></div></div>
</div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br>~~~~~~~~~~~~~~~~~~~~~~~~~~~<br>Dan Wendlandt <div>Nicira, Inc: <a href="http://www.nicira.com" target="_blank">www.nicira.com</a><br><div>twitter: danwendlandt<br>
~~~~~~~~~~~~~~~~~~~~~~~~~~~<br></div></div><br>