Open Stack

Nikola, thank you for the explanation.

So here is what we have problems with now (I might not be completely 
correct in some facts and guesses though):

The following review made it worse for us initially:
https://review.openstack.org/#/c/59034/25

As I understand in this review you replaced storing of bdmv1 with bdmv2. 
And the latter stopped storing the device name into snapshot.
First of all, the question is, how do you guarantee that the order of 
devices (not boot order but the vda, vdb, vdc... naming rather) will 
stay the same after restoring instance from such a snapshot. You must 
rely on an order they are stored in the image with respect to boot 
devices, right?
Another question, did it work for Xen fully before (honoring the names, 
setting them properly inside instance and reporting them)? And if it 
did, now with bdmv2 it did stop, right? If that's the case, is it the 
desired behavior?

For EC2 API, I presume, ideal situation would be if all backends could 
support device names fully. Since it is unrealistic, I guess, we'd still 
aim for it working fully where possible (Xen), setting up, assigning the 
same names to devices inside instance, i.e. complete synchronization of 
the outside and inside world. And for the rest to work in the most 
predictable way.
Do you think, since you're replicating the libvirt logic to order 
devices, you could give them some ordered names (a, b, c, d) according 
to the order and report them through public APIs and store into 
snapshots? This way we could use them to address these same devices in 
our use cases. And in case of Xen you can use real names instead.

For the reference, we have the following use cases concerned with device 
names (I'll be using EC2 API commands for use cases - they should be 
pretty much self-explanative, but you can look them up by the links for 
more descriptions):

1. RunInstance. Change parameters of devices during instance booting 
from image. In Grizzly it worked so we could specify changed BDM in 
parameters, it overwrote in nova DB the one coming from image and then 
started the instance with new parameters. The only key for addressing 
devices in this use case is the very device name. And now we don't have 
it for the volumes in BDM coming from the image, because nova stopped 
putting this information into the image.
http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_RunInstances.html

2. Devices names for Xen-backed instances to work fully. We should be 
able to specify required device names during initial instance creation, 
they should be stored into an image when the instance is shapshotted, we 
can fetch info and change parameters of such volume during subsequent 
operations, and the device names inside the instance should be named 
exactly.

3. DescribeInstances and DescribeInstanceAttributes to return BDM with 
device names ideally corresponding to actual device naming in instance.
http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeInstances.html
http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeInstanceAttribute.html

4. DescribeImages and DescribeImageAttributes to return BDM with device 
names ideally corresponding to the ones in instance before snapshotting.
http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeImages.html
http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeImageAttribute.html

5. AttachVolume with the specified device name.
http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_AttachVolume.html

6. ModifyInstanceAttribute with BDM as parameter.
http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_ModifyInstanceAttribute.html

7. ModifyImageAttribute with BDM as parameter.
http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_ModifyImageAttribute.html


Best regards,
   Alex Levine


On 5/27/15 8:55 PM, Nikola Đipanov wrote:
> On 05/27/2015 09:47 AM, Alexandre Levine wrote:
>> Hi all,
>>
>> I'd like to bring up this matter again, although it was at some extent
>> discussed during the recent summit.
>>
>> The problem arises from the fact that the functionality exposing device
>> names for usage through public APIs is deteriorating in nova. It's being
>> deliberately removed because as I understand, it doesn't universally and
>> consistently work in all of the backends. It happens  since IceHouse and
>> introduction of bdm v2. The following very recent review is one of the
>> ongoing efforts in this direction:
>> https://review.openstack.org/#/c/185438/
>>
> I've abandoned the change as it is clear we need to discuss how to go
> about this some more.
>
> But first let me try to give a bit more detailed explanation and
> background to what the deal is with device names. Supplying device names
> that will be honoured by the guests is really only possible with Xen PV
> guests (meaning the guest needs to be running PV-enabled kernel and
> drivers).
>
> Back in Havana, when we were working on [1] (see [2] for more details)
> the basic idea was that we will still accept device names because
> removing them from the public API is not likely to happen (mostly
> because of the EC2 compatibility), but in case of libvirt driver, we
> will treat them as hints only, and provide our own (by mostly
> replicating the logic libvirt uses to order devices [3]). We also
> allowed for device names to not be specified by the user as this is
> really what anyone not using the EC2 API should be doing (users using
> the EC2 API do however need to be aware the fact that i may not be
> honoured).
>
> [1]
> https://blueprints.launchpad.net/nova/+spec/improve-block-device-handling
> [2] https://wiki.openstack.org/wiki/BlockDeviceConfig
> [3]
> https://github.com/openstack/nova/blob/master/nova/virt/libvirt/blockinfo.py
>
>> The reason for my concern is that EC2 API have some important cases
>> relying on this information (some of them have no workarounds). Namely:
>> 1. Change of parameters set by image for instance booting.
>> 2. Showing instance's devices information by euca2ools.
>> 3. Providing additional volumes for instance booting
>> 4. Attaching volume
>> etc...
>>
> So based on the above - it seems to me that you think we are removing
> the information about device names completely. That's not the case -
> currently it is simply not mandatory for the Nova boot API call (it was
> never mandatory for volume attach afaict) - you can still pass it in,
> though libvirt may not honour it. It will still be tracked by the Nova
> DB and available for users to refer to.
>
>> Related to device names and additional related features we have troubles
>> with now:
>> 1. All device name related features
> As I said - they are not removed, in addition, you can still completely
> disregard the BDMv2 syntax as Nova should transparently handle old-style
> syntax when passed in (actually since BDM info is stored with images
> when snapshotting and it may have been v1 syntax - it is likely that we
> will never remove this support). If you are seeing some bugs related to
> this - please report them.
>
>> 2. Modification of deleteOnTermination flag
> I don't have enough details on this but if some behaviour has changed
> when using the old syntax - it is likely a bug so please report it.
>
>> 3. Modification of parameters for instance booting
> Again - I am not sure what this is related to exactly - but none of the
> parameters have changed really (only new ones were added). It would be
> good to get more information on this (preferably a bug report).
>
>> 4. deleteOnTermination and size of volume aren't stored into instance
>> snapshots now.
>>
> This does sound like a bug - and hopefully an easy to fix one.
>
>> Discussions during the summit on the matter were complicated because
>> nobody present really understood in details why and what is happening
>> with this functionality in nova. It was decided though, that overall
>> direction would be to add necessary features or restore them unless
>> there is something really showstopping:
>> https://etherpad.openstack.org/p/YVR-nova-contributor-meetup
>>
>> As I understand, Nikola Depanov is the one working on the matter for
>> some time obviously is the best person who can help to resolve the
>> situation. Nikola, if possible, could you help with it and clarify the
>> issue.
>>
>> My suggestion, based on my limited knowledge at the moment, still is to
>> restore back or add all of the necessary APIs and provide tickets or
>> known issues for the cases where the functionality is suffering from the
>> backend limitations.
>>
>> Please let me know what you think.
>>
> As explained above - nothing was intentionally removed, and if something
> broke - it's a bug that we should fix, so I urge the team behind the EC2
> API on stackforge to report those, and I will try to at least look into
> them, if not fix them. We might want to have a tag for EC2 related bugs
> in LP (I seem to remember there being such a thing before).
>
> Device names though are not something we can easily resolve without
> having the users of the EC2 API be aware that they may be talking to a
> Nova deployment, however I am unsure how much of a problem this is in
> practice, as I believe most users would have to be anyway.
>
> Hope this lifts some of the mystery around the topic - and I look
> forward to helping the EC2 efforts - but we really need to make sure we
> have the issues written down first :)
>
> N.
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev