Open Stack

I won't be present at the forthcoming Austin summit, so to prepare other
people in case there are f2f discussions, this is a rough status report
on the os-vif progress


os-vif core
-----------

NB by os-vif core, I mean the python packages in the os_vif/ namespace.

The object model for describing the various different VIF backend
configurations is defined well enough that it should cover all the
VIF types currently used by Nova libvirt driver, and probably all
those needed by other virt drivers. The only exception is that we
do not have a representation for the vmware 'dvs' VIF type. There's
no real reason why not, other than the fact that we're concentrating
on converting the libvirt nova driver first. These are dealt with
by the os_vif.objects.VIFBase object and its subclasses.


We now have an object model for describing client host capabilities.
This is dealt with by the os_vif.objects.HostInfo versioned object
and things is used. Currently this object provides details of all
the os-vif plugins that are installed on the host, and which VIF
configs objects each supports.  The intent is that the HostInfo
object is serialized to JSON, and passed to Neutron by Nova when
creating a port.  This allows Neutron to dynamically decide which
plugin and which VIF config it wants to use for creating the port.


The os_vif.PluginBase class which all plugins must inherit from
has been enhanced so that plugins can declare configuration
parameters they wish to support. This allows config options for
the plugins to be included directly in the nova.conf file in
a dedicated section per plugin. For example, the linux bridge
plugin will have its parameters in a "[os_vif_linux_bridge]"
section in nova.conf.  This lets us setup the deprecations
correctly, so that when upgrading from older Nova, existing
settings in nova.conf still apply to the plugins provided
by os-vif.


os-vif reference plugins
------------------------

Originally the intention was that all plugins would live outside
of the os-vif package. During discussions at the Nova mid-cycle
meeting there was a strong preference to have the linux bridge
and openvswitch plugin implementations be distributed as part of
the os-vif package directly.

As such we now have 'vif_plug_linux_bridge' and 'vif_plug_ovs'
python packages as part of the os-vif module. Note that these
are *not* under the os_vif python namespace, as the intention
was to keep their code structured as if they were separate,
so we can easily split them out again in future in we need to.

Both the linux bridge and ovs plugins have now been converted
over to use oslo.privsep instead of rootwrap for all the places
where they need to run privileged commands.


os-vif extra plugins
--------------------

Jay has had GIT repositories created to hold the plugins for all
the other VIF types the libvirt driver needs to support to have
feature parity with Mitaka and earlier. AFAIK, no one has done
any work to actually get the code for these working. This is not
a blocker, since the way the Nova integration is written allows
us to incrementally convert each VIF type over to use os-vif, so
we avoid need for a "big bang".


os-vif Nova integration
-----------------------

I have a patch up for review against Nova that converts the libvirt
driver to use os-vif. It only does the conversion for linux bridge
and openvswitch, all other vif types fallback to using the current
code, as mentioned above.  The unit tests for this pass locally,
but I've not been able to verify its working correctly when run for
real. There's almost certainly privsep related integration tasks to
shake out - possibly as little as just installing the rootwrap filter
needed to allow use of privsep. My focus right now is ironing this
out so that I can verify linux bridge + ovs work with os-vif.


There is a new job defined in the experimental queue that tests that
can verify Nova against os-vif git master so we can get forwarning
if something in os-vif will cause Nova to break. This should also
let us verify that the integration is actually working in Nova CI
before allowing it to actually merge.


os-vif Neutron integration
--------------------------

As mentioned earlier we now have a HostInfo versioned object defined
in os-vif which Nova will populate. We need to extend the Neutron API
to accept this object when nova creates a port. This lets Neutron know
which VIF plugins are available and the configs they require.

Once Neutron has this information, instead of sending back the current
unstructured port binding dict, it will be able to send back a serialized
os_vif.objects.VIFBase subclass which formally describes the VIF it wants
Nova to use. This might be possible by just defining a VIF_TYPE_OS_VIF
and putting the VIFBase serialized data in another port binding metadata
field. Alternatively it might be desirable to extend the Neutron API to
more explictly represent os-vif. 

None of the Neutron integration has been started, or even written up
since it is not a blocker for completing the Nova conversion. None the
less there is a strong desire from Neutron side to be able to dynamically
choose between different VIF types, so the sooner we start this the better.


Neuton work items
-----------------

Based on that the work items that we have for Neuton include

 - Get the initial port of Nova libvirt to use os-vif for linux bridge
   and openvswitch. The code is essentially written, just needs debugging
   and some magic to ensure it all runs correctly in the gate. This is
   my immediate personal task to complete

 - Convert more VIF types in nova libvirt to use os-vif with privsep.
   The GIT repos exist, and os-vif core team is setup as the core approvers
   for each repo. The intention, however, is that vendors would take
   responsiblity for each dealing with creating a os-vif plugin for
   their respective VIF types.  os-vif core team will provide whatever
   assistance is needed to smooth the process. While this is not a firm
   blocker, it is desirable if we can get all nova libvirt VIF types
   converted to os-vif for Neuton release.

 - Convert other Nova virt drivers to use os-vif. No work has been
   started on this. Again, expectation is that respective virt driver
   teams would want to lead this effort themselves. We're not going to
   force a big-bang conversion to os-vif for other virt drivers during
   Neuton.  We might want to consider mandating conversion to os-vif
   for Ocata though, since the longer we delay the conversion, the
   longer it'll be before we can deprecate & drop the old vif support
   code.

 - Convert Neutron to use os-vif for communication with Nova. The first
   step here is for someone on os-vif core team to write a spec and
   post it to neutron-specs, and get the Neutron devs engaged in the
   discussion. There will need to be a nova-spec too, but that's really
   just about Nova adopting whatever Neutron API changes are decided
   upon, so could be just a spec-less blueprint that points to the
   neutron spec for the details. No one has started on this, but I was
   intending todo so after the initial Nova/os-vif integration was
   working.


So the only real immediate blocker is getting the initial Nova/libvirt
os-vif integration functional for linux bridge + ovs, to prove that the
overall system is actually working as intended. Once that's merged
everything else can carry on with high degree of parallelization.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|