[openstack-dev] [nova] How should libvirt pools work with distributed storage drivers?

Solly Ross sross at redhat.com
Wed Nov 26 20:26:10 UTC 2014

> Some days ago, a bunch of Nova specs were approved for Kilo. Among them was
> https://blueprints.launchpad.net/nova/+spec/use-libvirt-storage-pools
> Now, while I do recognize the wisdom of using storage pools, I do see a
> couple of possible problems with this, especially in the light of my
> upcoming spec proposal for using StorPool distributed storage for the VM
> images.
> My main concern is with the explicit specification that the libvirt pools
> should be of the "directory" type, meaning that all the images should be
> visible as files in a single directory. Would it be possible to extend the
> specification to allow other libvirt pool types, or to allow other ways of
> pointing Nova at the filesystem path of the VM image?

The specification was never intended to restrict storage pools to being
file-based.  In fact, it was my intention that all different types of pools
be supported.  The specification dedicates several paragraphs to discussing
file-based pools, since transitioning from "legacy" file-based backends to
the storage pool backend requires a bit of work, while other backends, like
LVM, can simply be turned into a pool without any movement or renaming of the
underlying volumes.

In fact, LVM works excellently (it's one of the pool types I use frequently
in testing to make sure migration works regardless of source and destination
pool type).

> Where this is coming from is that StorPool volumes (which we intend to write
> a DiskImage subclass for) appear in the host filesystem as
> /dev/storpool/volumename special files (block devices). Thus, it would be...
> interesting... to find ways to make them show up under a specific directory
> (yes, we could do lots and lots of symlink magic, but we've been down that
> road before and it doesn't necessarily lead to Good Things(tm)). I see that
> the spec has several mentions of "yeah, we should special-case Ceph/RBD
> here, since they do things in a different way"- well, StorPool does things
> in a slightly different way, too :)

The reason that I wrote something about Ceph/RBD is that the Ceph storage driver
in libvirt is incomplete -- it doesn't yet have support for
virStorageVolCreateXMLFrom, so we need to work around that.

> And yes, we do have work in progress to expose the StorPool cluster's volumes
> as a libvirt pool, but this might take a bit of time to complete and then it
> will most probably take much more time to get into the libvirt upstream
> *and* into the downstream distributions, so IMHO "okay, let's use different
> libvirt pool types" might not be entirely enough for us, although it would
> be a possible workaround.

The intention was that new storage pool types should try to get themselves in
as new libvirt storage pool drivers, and then they should "just work" in Nova
(there is one line that needs to be modified, but other than that, you
should just be able to start using them).

> Of course, it's entirely possible that I have not completely understood the
> proposed mechanism; I do see some RBD patches in the previous incarnations
> of this blueprint, and if I read them right, it *might* be trivial to
> subclass the new libvirt storage pool support thing and provide the
> /dev/storpool/volumename paths to the upper layers. If this is so, feel free
> to let me know I've wasted your time in reading this e-mail, in strong terms
> if necessary :)

I dislike using "strong terms" ;-), but I do think you may have misread the spec.
If you are unclear, you can catch me next week on freenode as "directxman12" and
we can discuss further (I'm out on PTO this week).

Best Regards,
Solly Ross

More information about the OpenStack-dev mailing list