Open Stack

Hi Sumit,

This is a great question to be asking, as it gets to the heart of how
Quantum will work with Nova (or any any other OpenStack service that plugs
into the network).  As you suggest, our goal here is to try and come up with
a clean separation between the responsibilities of Nova and
the responsibilities of Quantum.

The way I think about it, the act of "attaching a VM NIC to the network"
actually happens at two distinct layers: logical (managed by Quantum) and,
for lack of a better name, non-logical (managed by Nova).

Non-Logical: This occurs when the Nova spins up a VM and plugs the VM NIC
into the vswitch.  For example, with Nova using libvirt, this amounts to
creating a linux device representing the VM NIC (e.g., tap0) and associating
it with a vswitch (e.g., br0).  With Quantum, this act alone should not
actually cause the VM to be able to send and receive packets, unless the
logical connectivity of the VM NIC has already been setup (see below).
 Instead, it is just establishing a channel by which vswitch controlled by
Quantum can pass packets to and from the VM NIC once the proper logical
connectivity has been specified.  With Quantum, Nova must have an
"interface-id" for each VM NIC.  When Nova spins up a VM and creates a new
port on a vswitch, it communicates to Quantum the interface-id associated
with that port on the vswitch.  We refer to as the "edge binding" (see
slides #7 and #14 in
http://www.slideshare.net/danwent/quantum-diablo-summary).

Logical: This type of "plugging" is done by the tenant via the Quantum API.
 The Quantum API exposes the concept of networks and ports on those networks
to model this logical connectivity.  Interface-ids can be "attached" to a
port, indicating the network connectivity that should be seen by the VM NIC
with the associated interface-id (e.g., two VMs attached to ports on the
same logical network can connect directly at L2, others cannot).  A simple
implementation of a logical network could be to put implement each logical
network as a different VLAN within a single physical L2 domain, with a
Quantum plugin managing the vswitch to put each port on the appropriate VLAN
(see slide #15).

The motivation for this design is that Nova does not need to be involved
when the logical connectivity of a VM changes. Nova just needs to worry
about making sure the vswitch (or physical switch if using VNTag/VEPA)
managed by the Quantum plugin has a channel to pass bits to and from each VM
NIC, and knows the identify of each of those channels (the edge binding).
 It is up to the Quantum plugin managing the vswitch to make sure the right
logical connectivity (or lack thereof) is implemented.

Along with folks from the Nova team, we've been discussing the details of
how this model applies to different Nova "virt" layer implementations,
including libvirt, XenServer, VMware, etc.  As you suggest, the libvirt
model where Nova specifies only a bridge to connect to is less flexible, so
we're looking at using an <interface> elements of type "ethernet" instead of
type "bridge" as an alternative (Ryu did some work on this in a branch).

Implementation work on an experimental version of Quantum has been
progressing well, so in another week or two I would expect that we can have
a full stack of code that people can play with using a slightly tweaked
version of Nova.  Having this available will hopefully make the above
description of a logical + non-logical plugging step more concrete and will
serve as a good vehicle for feedback from the community.  Happy to chat more
about this tomorrow at the Networking meeting on IRC.

dan



On Mon, May 30, 2011 at 9:01 PM, Sumit Naiksatam (snaiksat) <
snaiksat at cisco.com> wrote:

> Hi,
>
>
>
> I would like to request clarity on the semantics of the "attach resource to
> port" operation proposed in Quantum in the context of spawning a VM. Let's
> discuss this with respect to a specific example. As of today, nova-compute
> generates the domain configuration (libvirt.xml), including the network
> interface configuration as below -
>
>
>
>         <interface type='bridge'>
>
>             <source bridge='br100'/>
>
>             <mac address='02:16:3e:78:4d:84'/>
>
>             <!--   <model type='virtio'/>  CANT RUN virtio network right
> now -->
>
>             <filterref filter="nova-instance-instance-00000001">
>
>                 <parameter name="IP" value="10.1.0.2" />
>
>                 <parameter name="DHCPSERVER" value="10.1.0.1" />
>
>                 <parameter name="PROJNET" value="10.1.0.0" />
>
>                 <parameter name="PROJMASK" value="255.255.255.128" />
>
>             </filterref>
>
>         </interface>
>
>
>
> By virtue of specifying "br100" as the bridge to connect to, nova-compute
> is configuring the attachment from the virtual NIC (on the VM) to the
> network, and the "attachment" operation is (implicitly) realized when the VM
> is spawned (and running).
>
>
>
> With the introduction of the "attach resource to port" operation, we are
> introducing an explicit step in the process of the VM spawning. It implies
> that prior to invoking the "attach resource to port" operation, nova-compute
> would not know which bridge to connect to (i.e. it would not know the
> identity of "br100"). So does nova-compute first spawn the VM without the
> network interface configuration? If that is the case, it brings us to the
> next question, which is, what does the "attach resource to port" operation
> actually do? There are at least two possibilities here -
>
>
>
> (1) The operation merely results in the identification (and, if required,
> the configuration) of "br100" as the bridge to connect the VM's network
> interface to. If that is the case, we need a way to propagate this
> information (i.e. "br100") back to the entity which invoked the "attach
> resource to port" operation. Subsequently, that same entity will have to
> perform another action to craft the VM's network interface configuration
> (with the acquired knowledge of "br100") and also add this network interface
> to the configuration of the VM (by making appropriate libvirt calls). Adding
> this network interface configuration could be done, both, prior to starting
> the VM, or even while the VM is running.
>
>
>
> OR
>
>
>
> (2) The operation not only identifies/configuration of "br100", but also
> crafts the VM's network interface configuration, and adds that interface to
> the VM (by making appropriate libvirt calls). Note that this second option
> amounts to Quantum controlling a resource on the VM (which ideally should be
> outside the scope of Quantum).
>
>
>
> OR, is it something else?
>
>
>
> Kindly let me know what you thoughts are on this. As projected above, these
> semantics have implications on how nova-compute will deal with spawning VMs
> (and it might be different from what it is today), and hence it might be
> important to address this issue sooner.
>
>
>
> Thanks,
>
> ~Sumit.
>
> PS: As I write this, it occurs to me that it's not very clear as to what
> the "create port" operation would do in the above case as well, since no
> actual port is created on the bridge. Any thoughts?
>
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack at lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>


-- 
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Dan Wendlandt
Nicira Networks, Inc.
www.nicira.com | www.openvswitch.org
Sr. Product Manager
cell: 650-906-2650
~~~~~~~~~~~~~~~~~~~~~~~~~~~
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20110530/0ff95d5b/attachment.html>