[openstack-dev] [nova][cinder] How will nova advertise that volume multi-attach is supported?
mriedem at linux.vnet.ibm.com
Wed Jan 13 22:52:37 UTC 2016
tl;dr - do we need a REST API microversion for multi-attach support in nova?
The volume multi-attach series in nova, starting here , has run into
an upgrade problem.
Basically, there is code in liberty which doesn't pass an instance uuid
and volume id to the block_device_mappings table query to uniquely
identify BDMs for operations like detach, swap volume and live
migration. This is a problem when we are trying to introduce support for
volume multiattach because today nova considers volume/instance as a 1:1
relationship, but this introduces a 1:M relationship (oh, and the nova
bdm table doesn't have any unique constraints enforcing data integrity -
So in Mitaka we are for sure going to land the code that makes sure that
the operations which have a volume and instance get a BDM based on those
keys. The only one that doesn't is assisted-snapshots, which is a REST
API limitation, and we're going to block that case from multiattach
support (when it lands). Dan Smith is starting that cleanup work here .
Ildiko has been asking how multiattach could be merged in nova in mitaka
and actually supported in the REST API, DB API, compute and virt driver
layers. Because of this query issue, and rolling upgrades, we could have
a mitaka API talking to liberty computes. If we allow sending a
multiattach volume to a liberty compute that already has attachments,
that will fail.
We talked about adding a check in the API layer to see if there are any
liberty computes running and if not, then we could allow the 2nd attach
on a multi-attach volume, but that is racy and not fail-safe, which
could mean a user could get a multi-attach request to pass in one case
but fail in another (it's a problem related to how the service versions
are cached in-memory per API worker and isn't something that can be
resolved before mitaka release).
Ildiko was asking about adding a configuration option to the compute API
which basically toggles the multi-attach feature. The thinking is that
would default to False in mitaka and an operator would flip it to True
once all the computes are upgraded to mitaka. We don't normally
feature-toggle like this though, so it's not an attractive option really.
A lot of this also falls down in the aspect of how a user actually knows
when the compute endpoint they are talking to supports multi-attach. If
you're trying to attach a volume to a 2nd instance after it's already
attached today, you'll get a 400 response.
If we did the config option idea and it was False, you'd get a 400
response for that also, even if you were running mitaka-level compute
API (but still had liberty computes).
Normally we advertise capability in the API via microversions. So I'm
thinking what we need is basically a microversion in the volume attach
REST API which says:
1. if you have a multiattach=True volume
2. that is already attached to instance 1
3. and you're trying to attach it to instance 2
4. that fails *unless* you're requesting a new enough microversion
(opt-in to the feature).
That seems clear enough from the client perspective I think.
There is another wrinkle, however, which is not unique to this case.
There are only certain cinder backends (lvm) and nova virt drivers
(libvirt) that will support multiattach. In the case of nova, you don't
know if the multiattach will actually be accepted until we've cast off
to the compute and find out if the virt driver supports it.
So even if we have a microversion in the REST API, and all of your
computes are at >=mitaka, if you're hitting a non-libvirt compute node,
it will fail (you get an instance fault in the case of volume attach,
and a NoValidHost in the case of boot from volume).
I don't think we have a solution to that issue right now. I think John
has been trying to sort that out somehow with the feature classification
effort, but I'm unclear on the details.
But I think the first question is whether or not we require a
microversion for multiattach support in the REST API purely as a signal
to client code that a given cloud supports it, at least on certain nodes
(running lvm + libvirt + mitaka).
More information about the OpenStack-dev