[openstack-dev] [nova] [neutron] PCI pass-through network support

Ian Wells ijw.ubuntu at cack.org.uk
Thu Jan 16 00:28:58 UTC 2014

To clarify a couple of Robert's points, since we had a conversation earlier:
On 15 January 2014 23:47, Robert Li (baoli) <baoli at cisco.com> wrote:

>   ---  do we agree that BDF address (or device id, whatever you call it),
> and node id shouldn't be used as attributes in defining a PCI flavor?

Note that the current spec doesn't actually exclude it as an option.  It's
just an unwise thing to do.  In theory, you could elect to define your
flavors using the BDF attribute but determining 'the card in this slot is
equivalent to all the other cards in the same slot in other machines' is
probably not the best idea...  We could lock it out as an option or we
could just assume that administrators wouldn't be daft enough to try.

        * the compute node needs to know the PCI flavor. [...]
>                   - to support live migration, we need to use it to create
> network xml

I didn't understand this at first and it took me a while to get what Robert
meant here.

This is based on Robert's current code for macvtap based live migration.
The issue is that if you wish to migrate a VM and it's tied to a physical
interface, you can't guarantee that the same physical interface is going to
be used on the target machine, but at the same time you can't change the
libvirt.xml as it comes over with the migrating machine.  The answer is to
define a network and refer out to it from libvirt.xml.  In Robert's current
code he's using the group name of the PCI devices to create a network
containing the list of equivalent devices (those in the group) that can be
macvtapped.  Thus when the host migrates it will find another, equivalent,
interface.  This falls over in the use case under consideration where a
device can be mapped using more than one flavor, so we have to discard the
use case or rethink the implementation.

There's a more complex solution - I think - where we create a temporary
network for each macvtap interface a machine's going to use, with a name
based on the instance UUID and port number, and containing the device to
map.  Before starting the migration we would create a replacement network
containing only the new device on the target host; migration would find the
network from the name in the libvirt.xml, and the content of that network
would behave identically.  We'd be creating libvirt networks on the fly and
a lot more of them, and we'd need decent cleanup code too ('when freeing a
PCI device, delete any network it's a member of'), so it all becomes a lot
more hairy.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140116/93f065e6/attachment.html>

More information about the OpenStack-dev mailing list