Hi Daniel,<div><br></div><div>Ok, I think we're zeroing in on the key differences in our points of view. Let me know if you think it would be more efficient to discuss this via IRC, as email lag when we're on different sides of the globe is tough. More comments inline.</div>
<div><br></div><div><br><div class="gmail_quote">On Wed, Jan 23, 2013 at 3:02 AM, Daniel P. Berrange <span dir="ltr"><<a href="mailto:berrange@redhat.com" target="_blank">berrange@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="HOEnZb"><div class="h5">On Tue, Jan 22, 2013 at 11:41:55PM -0800, Dan Wendlandt wrote:<br>
> Hi Daniel,<br>
><br>
> More comments inline.<br>
><br>
> On Wed, Jan 16, 2013 at 1:48 AM, Daniel P. Berrange <<a href="mailto:berrange@redhat.com">berrange@redhat.com</a>>wrote:<br>
><br>
> > On Tue, Jan 15, 2013 at 10:50:55PM -0800, Dan Wendlandt wrote:<br>
> > > > As an example, when configuring OpenVSwitch one of the pieces of<br>
> > > > information<br>
> > > > require is an 'interfaceid'. Previously the libvirt VIF driver set the<br>
> > > > interfaceid<br>
> > > > based on the vif['uuid'] field, since that is what quantum expected.<br>
> > This<br>
> > > > is not<br>
> > > > portable though. The correct approach is for nova.network.model to<br>
> > have an<br>
> > > > explicit 'ovs-interfaceid' field and the nova.network.quantumv2.api<br>
> > driver<br>
> > > > sets this based on the on vif['uuid']. The libvirt VIF driver can now<br>
> > > > configure<br>
> > > > OVS with any network driver, not simply Quantum. Similarly for things<br>
> > like<br>
> > > > the<br>
> > > > OVS bridge name, of the TAP device names, all of which previously had<br>
> > to be<br>
> > > > hardcoded to Quantum specific data. This extends to bridging, 801.qbh,<br>
> > > > etc,etc<br>
> > > ><br>
> > ><br>
> > > I have no real concerns here with respect to these representations within<br>
> > > Nova. Since you mentioned that you view this as not being libvirt<br>
> > > specific, would the VIF/Network model in nova/network/model.py be<br>
> > extended<br>
> > > with additional fields not currently represented in the spec (<br>
> > > <a href="http://wiki.openstack.org/LibvirtVIFDrivers" target="_blank">http://wiki.openstack.org/LibvirtVIFDrivers</a>) for platforms like<br>
> > XenServer<br>
> > > (plug into XenServer network UUIDs), vmware (plug into port group-id), a<br>
> > > hyper-v virtual network, etc?<br>
> ><br>
> > Sure, if we find additional pieces of information are needed in the models,<br>
> > we may have to extend them further. I'm surprised that QUantum server<br>
> > actually<br>
> > has knowledge of the hypervisor specific concepts you mention above though.<br>
> ><br>
><br>
> I'm confused, as my understanding of your proposal requires Quantum know<br>
> this type of information. Perhaps I'm missing something? From the code<br>
> review, you have comments like:<br>
><br>
> # TODO(berrange) temporary hack until Quantum can pass over the<br>
> # name of the OVS bridge it is configured with<br>
><br>
> and<br>
><br>
> # TODO(berrange) Quantum should pass the bridge name<br>
> # in another binding metadata field<br>
><br>
> In this case, the name of the bridge is a concept specific to certain linux<br>
> hypervisors. If I was using XenServer, the equivalent might be a Network<br>
> UUID, or with ESX a port group uuid.<br>
><br>
> My current thinking is that Quantum shouldn't have to know such information<br>
> either, but based on your comments, I was assuming this was a point of<br>
> disagreement. Can you clarify?<br>
<br>
</div></div>Actually the Xen OVS VIF driver does require a bridge name. The bridge<br>
name is then used by the VIF driver to lookup the Xen network UUID. </blockquote><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
So this is a good example of the same information being required by<br>
multiple hypervisor drivers. Similarly the Xen OVS VIF driver also<br>
requires the ovs interfaceid - again it currently hardcodes the<br>
assumption the the interfaceid is based on the vif['id'] field.<br>
So again my chage to the VIF model to include an explicit ovs_interfaceid<br>
parameter makes sense for Xen too.<br></blockquote><div><br></div><div>I completely agree that some fields make sense to multiple hypervisors... I certainly did not intend to say anything to the contrary. The point I was making was that there is no single set of information is relevant to all hypervisors. Do you agree with that statement, or are you advocating that there is a single set of such information? </div>
<div><br></div><div>Also, I'm still trying to get confirmation to my question above, namely that you do intend that Quantum would provide all such data needed to plug a VIF, for example, providing a bridge name to a hypervisor running KVM, or a port-group id for a hypervisor running ESX. </div>
<div><br></div><div>If so, I do not see how the existing proposal can handle the situation where two different sets of hypervisors that need different information are deployed simultaneously. This could happen either because the two sets of hypervisors identify vswitches differently (e.g., linux bridge names vs. port-group ids) or because deployment constraints make it impossible to use the same vswitch identifier across all hypervisors (e.g., a vswitch is identified by a uuid, but that UUID is a per-cluster ID and the deplyoment has multiple clusters). Helping me understand how you see this working would help me out a lot. </div>
<div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im"><br>
> > Surely over time nova will have to add<br>
> > > entirely new vif_types. Since all Nova code for a given deployment will<br>
> > be<br>
> > > in sync, this is not a problem within Nova, but what if you deploy a<br>
> > newer<br>
> > > Quantum with an older Nova? Let's say in "H" Nova introduces vif_type<br>
> > > "qbx" (presumably some "improved" for of VEPA :P) with requires a<br>
> > different<br>
> > > set of parameters to configure. If a tenant is running an "H" version of<br>
> > > Quantum, with a Grizzly version of Nova, how does it know that it can't<br>
> > > specify vif-type qbx?<br>
> ><br>
> > I mostly think that this is a documentation / deployment issue for<br>
> > admins to take care of. In the scenario you describe if you have an<br>
> > Hxxxx Quantum running with a Grizzly Nova, the admin should expect<br>
> > that they won't neccessarily be able to use latest pieces of Quantum.<br>
> ><br>
><br>
> I think we both agree that using vif_type qbx would not work (and that this<br>
> is reasonable). That wasn't my question though. My question was: If<br>
> Quantum returns qbx, presumably the older Nova would error-out when<br>
> provisioning the VM, so how does Quantum from the "H" series know that it<br>
> is or is not OK to return vif-type "qbx"? If we have to have an admin<br>
> configure quantum with the set of vif_types the Nova install supports,<br>
> where back to what we wanted to avoid: having to have an admin sync config<br>
> between Nova + Quantum.<br>
<br>
</div>You seem to be considering the case of a new vif_type, which replaces a<br>
previously used vif_type when upgrading Quantum. This is something which<br>
should never be done, unless Quantum wants to mandate use of a newer Nova. </blockquote><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
If you want to retain full version compatibility in all directions, then<br>
when upgrading Quantum, the existing information should never change,<br>
unless the admin has explicitly chosen to reconfigure the plugin in some<br>
way.<br></blockquote><div><br></div><div><br class="Apple-interchange-newline">I don't think that's the use case I'm trying to describe. If a Quantum plugin only supports one vif_type for a given release, its trivial for it to know which one to respond with. I'm talking about the case of a Quantum plugin that supports two different vif_types simultaneousl, but needs to know information about what vif_types the corresponding nova-compute supports in order to know how to respond. I believe some other Red Hat folks are keen on being able to realize this. </div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
What is more likely is that some brand new Quantum plug is invented which<br>
also invents a new vif_type. This doesn't really require any special config<br>
handling. It is simply a documentation task to say that if you want to use<br>
the new plugin, you also need to use a new Nova.<br></blockquote><div><br></div><div>Yeah, this is the easy case. I have no concerns there. </div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im"><br>
> > > The above items are possible with the existing vif-plugging, since<br>
> > > different config flags can be passed to different nova-compute instances.<br>
> > ><br>
> > > It seems like in the new model, we would need Nova to pass some<br>
> > information<br>
> > > to Quantum so that Quantum can make the right decision about what<br>
> > vif-type<br>
> > > and data to send.<br>
> ><br>
> > Since the formal dependancy is really Nova -> Quantum, IMHO we should<br>
> > really<br>
> > just document that Nova must be at least as new as Quantum. Runing a Hxxxx<br>
> > Nova against a Grizzly Quantum should present no problems, only the reverse<br>
> > has issues.<br>
> ><br>
><br>
> This isn't just about old vs. new versions. See the example above about a<br>
> deployment that spans multiple cluster for a hypervisor like XenServer or<br>
> ESX, thereby requiring that a different identifier is passed back based on<br>
> which cluster. Or an even more extreme example (but one I've already seen<br>
> in production with Quantum) is that there are multiple hypervisor types, so<br>
> even the vif_type that would need to be passed back may well be different<br>
> on different hypervisors.<br>
<br>
</div>I don't think the vif_type should ever need to be different for different<br>
hypervisors. The vif_type is describing how Quantum has setup the network.<br>
Based off this, the hypervisor VIF drivers then decide how to configure<br>
the hypervisor. You seem to be thinking of the vif_type as a description<br>
of how the hypervisors must configure the network, which is backwards to<br>
how it should be.<br></blockquote><div><br></div><div>Yes, I think this is where a lot of the difference of opinion lies. To me the key question is whether you would use the same vif_type to plug a VM into an OVS bridge and into an ESX port group (assuming you had a quantum plugin that could both manage OVS and the vmware vswitch)? And even if you did, I feel that the identifiers that quantum would need to return would be different. Or are you suggesting that something like a port-group would be modeled as a "bridge"? It seems pretty clear from this commit (<a href="https://review.openstack.org/#/c/19117/5/nova/network/model.py">https://review.openstack.org/#/c/19117/5/nova/network/model.py</a>) that bridge means a linux bridge, as there's a variable that describes the length of the bridge name as the length of a linux device name. </div>
<div><br></div><div>I think if we can get to the bottom of this use case, we'll be well on our way to being on the same page. </div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im"><br>
> > It is a subtle distinction, but I guess I don't think of the data as soley<br>
> > being "what is the required VIF configuration", but rather "what<br>
> > information<br>
> > is required todo VIF configuration". As such knowing whether Quantum has<br>
> > applied filtering falls within scope.<br>
> ><br>
><br>
> Yes, but 'filtering' as a term seems very broad. Here's a made-up example:<br>
> what if the quantum plugin performs L3/L4 filtering (e.g., security groups)<br>
> but did not have the ability to prevent mac/arp spoofing, and instead<br>
> wanted the virt layer to handle that. To me, having values that explicitly<br>
> state what Quantum wants the virt layer to do would be more clear and less<br>
> likely to run into problems in the future. I'm guessing the current<br>
> definition is use to map closely to whether to push down any <filterref><br>
> arguments to libvirt? I'm just trying to think about how the concept would<br>
> apply to other platforms.<br>
<br>
</div>Given the example above, then we should not return a 'filtered: true/false'<br>
value, but rather return more fine grained data l3_filter: true/false,<br>
l4_flter:true/false, etc, etc. Nova can then decide what additional filtering<br>
is required, if any.<br>
<br>
The key point is that we should not be expecting the administrator to try<br>
to figure out what type of filtering a particular quantum plugin has applied<br>
& then configure nova manually. This implies the admin has to have far too<br>
much detailed knowledge of Quantum implementation details, which is just<br>
not acceptable. Quantum needs to inform Nova so it can do the right thing.<br></blockquote><div><br></div><div>I completely agree :)</div><div><br></div><div>Dan</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="HOEnZb"><div class="h5"><br>
<br>
Daniel<br>
--<br>
|: <a href="http://berrange.com" target="_blank">http://berrange.com</a> -o- <a href="http://www.flickr.com/photos/dberrange/" target="_blank">http://www.flickr.com/photos/dberrange/</a> :|<br>
|: <a href="http://libvirt.org" target="_blank">http://libvirt.org</a> -o- <a href="http://virt-manager.org" target="_blank">http://virt-manager.org</a> :|<br>
|: <a href="http://autobuild.org" target="_blank">http://autobuild.org</a> -o- <a href="http://search.cpan.org/~danberr/" target="_blank">http://search.cpan.org/~danberr/</a> :|<br>
|: <a href="http://entangle-photo.org" target="_blank">http://entangle-photo.org</a> -o- <a href="http://live.gnome.org/gtk-vnc" target="_blank">http://live.gnome.org/gtk-vnc</a> :|<br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br>~~~~~~~~~~~~~~~~~~~~~~~~~~~<br>Dan Wendlandt <div>Nicira, Inc: <a href="http://www.nicira.com" target="_blank">www.nicira.com</a><br><div>twitter: danwendlandt<br>
~~~~~~~~~~~~~~~~~~~~~~~~~~~<br></div></div>
</div>