Open Stack

On Fri, Jan 8, 2016 at 8:31 AM, Flavio Percoco <flavio at redhat.com> wrote:
> On 29/12/15 07:41 -0600, Sam Matzek wrote:
>>
>> On Thu, Dec 24, 2015 at 7:49 AM, Mikhail Fedosin <mfedosin at mirantis.com>
>> wrote:
>>>
>>> Hello, it's another topic about glance v2 adoption in Nova, but it's
>>> different from the others. I want to declare that there is a set of
>>> commits,
>>> that make Nova version agnostic and allow to work with both glance apis.
>>> The
>>> idea of the solution is to determine the current api version in the
>>> beginning and make appropriate requests after.
>>> (https://review.openstack.org/#/c/228578/,
>>> https://review.openstack.org/#/c/238309/,
>>> https://review.openstack.org/#/c/259097/)
>>>
>>> Indeed, it requires some additional (and painful) work, but now all
>>> tempest
>>> tests pass in Jenkins.
>>>
>>> Note: this thread is not about xenplugin, there is another topic, called
>>> 'Xenplugin + Glance_v2 = Hate'
>>>
>>> Here the main issues we faced and how we've solved them:
>>>
>>> 1. "changes-since" filter for image-list is not supported in v2 api.
>>> Steve
>>> Lewis made a great job and implemented a set of filters with comparison
>>> operators https://review.openstack.org/#/c/197388/. Filtering by
>>> 'changes-since' is completely similar to 'gte:updated_at'.
>>>
>>> 2. Filtering by custom properties must have prefix 'property-'. In v2
>>> it's
>>> not required.
>>>
>>> 3. V2 states that all custom properties must be image attributes, but
>>> Nova
>>> assumes that they are included in 'properties' dictionary. It's fixed
>>> with
>>> glanceclient's method 'is_base_property(prop_name)', that returns False
>>> in
>>> case of custom property.
>>>
>>> 4. is_public=True/False is visibility="public"/"private" respectively.
>>>
>>> 5. Deleting custom image properties in Nova is performed with
>>> 'purge_props'
>>> flag. If it's set to True, then all prop names, that are not included in
>>> updated data, will be removed. In case of v2 we have to explicitly
>>> specify
>>> prop names in the list param 'remove_props'. To implement this behaviour,
>>> if
>>> 'purge_props' is set, we make additional 'get' request to determine the
>>> existing properties and put in 'remove_prop' list only those, that are
>>> not
>>> in updated_data.
>>>
>>> 6. My favourite:
>>> There is an ability to activate an empty image by just providing 'size =
>>> 0':
>>> https://review.openstack.org/#/c/9715/, in this case image will be a
>>> collection of metadata. Glance v2 doesn't support this "feature" and
>>> that's
>>> why we have to implement a very dirty workaround:
>>>     * v2 requires, that disk_format and container-format must be set
>>> before
>>> the activation. if these params are not provided to 'create' method then
>>> we
>>> hardcode it to 'qcow2' and 'bare'.
>>>     * we call 'upload' method with empty data (data = '') to activate
>>> image.
>>> Me (and the rest glance team) think that this image activation with
>>> zero-size is inconsistent and we won't implement it in v2. So, I'm going
>>> to
>>> ask if Nova really needs this feature and maybe it's possible to make it
>>> work without it.
>>
>>
>> Nova uses this functionality when it creates snapshots of volume
>> backed instances, that is, instances that only have Cinder volumes
>> attached and do not have an ephemeral disk.
>> In this case Nova API creates Cinder snapshots for the Cinder volumes
>> and builds the block_device_mapping property with block devices that
>> reference the Cinder snapshots.  Nova activates this image with size=0
>> because this image does not have a disk and simply refers to the
>> collection of Cinder snapshots that collectively comprise the binary
>> image.  Callers of Glance outside of Nova may also use the APIs to
>> create "block device mapping" images as well that contain references
>> to Cinder volumes to attach, blank volumes to create, snapshots to
>> create volumes from, etc during the server creation.  Not having the
>> ability to create these images with V2 is a loss of function.
>
>
> I disagree. Being able to activate empty images breaks the consistency
> of Glances API and I don't think it should be considered a feature but
> a bug. An active image is an image that can be used to *boot* a VM. If
> the image is empty, you simply can't do that.
>
> Note that pointing to a block device makes the image "non-empty"
> already. There's a spec merged[0] and code written[1] to allow for
> using cinder as backend for Glance.
>
> If there's a missing functionality for the mapping users require, I
> believe activating empty images is not the solution for it.
>
> [0] https://review.openstack.org/183363
> [1] https://review.openstack.org/166414
>
>> The callers could supply 1 byte of junk data, like a space character,
>> but that would result in a junk image being stored in Glance's default
>> backing store for the image.  It would also give the impression that a
>> real disk image exists for the image in a backing store which could be
>> fetched, which is incorrect.
>>
>> If I remember correctly Glance V2 lets you transition an image from
>> queued to active state with size = 0 if the image is using an external
>> backing store such as http.  The store is then called to fetch the
>> size.  Would it be possible to allow Glance to consider images with
>> size = 0 and the block_device_mapping property to be "externally
>> sourced" and allow the transition?
>
>
> In v2 you can create an image and then add a location for it (this
> should be enabled, of course). You can even use cinder locations since
> Kilo (or Havana?).

I don't believe Cinder backing store / Cinder locations can work for
the Nova use case here.  In the case of this use case, Nova snapshot
of Cinder backed VM, Nova creates Cinder snapshots of every attached
Cinder volume and puts the snapshots in a BDM list in the Glance
metadata, and moves the image to Active with size = 0 and no volume.

This inherently creates an image that is pointing at a collection of
multiple image binaries (Cinder snapshots) that collectively make up
the "image".  For example the Cinder snapshots could be snapshot0 =
boot disk, snapshot1 = application binaries, snapshot1 = DB volume 1
snapshot2 = DB volume 2.

I may be wrong, but my understanding of multiple location support in
Glance is that the multiple locations are intended to all contain
copies / mirrors of the same image binary (singular), and the image
consumer, say, a Nova virt driver, could look at the locations and
make the best choice of location based on its settings for ephemeral
disk backing, etc.  If this is the correct understanding then the
multiple locations support using Cinder the Cinder backing store to
reference multiple Cinder snapshots of different Cinder volumes is not
a good fit.

Now possibly a compromise could exist where in the "snapshot volume
backed VM" flow, Nova changes to set location of the Glance image to
be the Cinder snapshot of the boot disk / root BDM, and then puts the
non-root BDM volumes in the block_device_mapping image metadata.

This would make the image non-empty and would use the Cinder backing
store for the boot disk while maintaining the existing functionality
of having the image refer to the other Cinder snapshots in the block
device mapping metadata.



>
> Cheers,
> Flavio
>
> --
> @flaper87
> Flavio Percoco
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>