Open Stack

On Fri, Jan 8, 2016 at 11:54 AM, Sam Matzek <matzeksam at gmail.com> wrote:
> On Fri, Jan 8, 2016 at 8:31 AM, Flavio Percoco <flavio at redhat.com> wrote:
>> On 29/12/15 07:41 -0600, Sam Matzek wrote:
>>>
>>> On Thu, Dec 24, 2015 at 7:49 AM, Mikhail Fedosin <mfedosin at mirantis.com>
>>> wrote:
>>>>
>>>> Hello, it's another topic about glance v2 adoption in Nova, but it's
>>>> different from the others. I want to declare that there is a set of
>>>> commits,
>>>> that make Nova version agnostic and allow to work with both glance apis.
>>>> The
>>>> idea of the solution is to determine the current api version in the
>>>> beginning and make appropriate requests after.
>>>> (https://review.openstack.org/#/c/228578/,
>>>> https://review.openstack.org/#/c/238309/,
>>>> https://review.openstack.org/#/c/259097/)
>>>>
>>>> Indeed, it requires some additional (and painful) work, but now all
>>>> tempest
>>>> tests pass in Jenkins.
>>>>
>>>> Note: this thread is not about xenplugin, there is another topic, called
>>>> 'Xenplugin + Glance_v2 = Hate'
>>>>
>>>> Here the main issues we faced and how we've solved them:
>>>>
>>>> 1. "changes-since" filter for image-list is not supported in v2 api.
>>>> Steve
>>>> Lewis made a great job and implemented a set of filters with comparison
>>>> operators https://review.openstack.org/#/c/197388/. Filtering by
>>>> 'changes-since' is completely similar to 'gte:updated_at'.
>>>>
>>>> 2. Filtering by custom properties must have prefix 'property-'. In v2
>>>> it's
>>>> not required.
>>>>
>>>> 3. V2 states that all custom properties must be image attributes, but
>>>> Nova
>>>> assumes that they are included in 'properties' dictionary. It's fixed
>>>> with
>>>> glanceclient's method 'is_base_property(prop_name)', that returns False
>>>> in
>>>> case of custom property.
>>>>
>>>> 4. is_public=True/False is visibility="public"/"private" respectively.
>>>>
>>>> 5. Deleting custom image properties in Nova is performed with
>>>> 'purge_props'
>>>> flag. If it's set to True, then all prop names, that are not included in
>>>> updated data, will be removed. In case of v2 we have to explicitly
>>>> specify
>>>> prop names in the list param 'remove_props'. To implement this behaviour,
>>>> if
>>>> 'purge_props' is set, we make additional 'get' request to determine the
>>>> existing properties and put in 'remove_prop' list only those, that are
>>>> not
>>>> in updated_data.
>>>>
>>>> 6. My favourite:
>>>> There is an ability to activate an empty image by just providing 'size =
>>>> 0':
>>>> https://review.openstack.org/#/c/9715/, in this case image will be a
>>>> collection of metadata. Glance v2 doesn't support this "feature" and
>>>> that's
>>>> why we have to implement a very dirty workaround:
>>>>     * v2 requires, that disk_format and container-format must be set
>>>> before
>>>> the activation. if these params are not provided to 'create' method then
>>>> we
>>>> hardcode it to 'qcow2' and 'bare'.
>>>>     * we call 'upload' method with empty data (data = '') to activate
>>>> image.
>>>> Me (and the rest glance team) think that this image activation with
>>>> zero-size is inconsistent and we won't implement it in v2. So, I'm going
>>>> to
>>>> ask if Nova really needs this feature and maybe it's possible to make it
>>>> work without it.
>>>
>>>
>>> Nova uses this functionality when it creates snapshots of volume
>>> backed instances, that is, instances that only have Cinder volumes
>>> attached and do not have an ephemeral disk.
>>> In this case Nova API creates Cinder snapshots for the Cinder volumes
>>> and builds the block_device_mapping property with block devices that
>>> reference the Cinder snapshots.  Nova activates this image with size=0
>>> because this image does not have a disk and simply refers to the
>>> collection of Cinder snapshots that collectively comprise the binary
>>> image.  Callers of Glance outside of Nova may also use the APIs to
>>> create "block device mapping" images as well that contain references
>>> to Cinder volumes to attach, blank volumes to create, snapshots to
>>> create volumes from, etc during the server creation.  Not having the
>>> ability to create these images with V2 is a loss of function.
>>
>>
>> I disagree. Being able to activate empty images breaks the consistency
>> of Glances API and I don't think it should be considered a feature but
>> a bug. An active image is an image that can be used to *boot* a VM. If
>> the image is empty, you simply can't do that.
>>
>> Note that pointing to a block device makes the image "non-empty"
>> already. There's a spec merged[0] and code written[1] to allow for
>> using cinder as backend for Glance.
>>
>> If there's a missing functionality for the mapping users require, I
>> believe activating empty images is not the solution for it.
>>
>> [0] https://review.openstack.org/183363
>> [1] https://review.openstack.org/166414
>>
>>> The callers could supply 1 byte of junk data, like a space character,
>>> but that would result in a junk image being stored in Glance's default
>>> backing store for the image.  It would also give the impression that a
>>> real disk image exists for the image in a backing store which could be
>>> fetched, which is incorrect.
>>>
>>> If I remember correctly Glance V2 lets you transition an image from
>>> queued to active state with size = 0 if the image is using an external
>>> backing store such as http.  The store is then called to fetch the
>>> size.  Would it be possible to allow Glance to consider images with
>>> size = 0 and the block_device_mapping property to be "externally
>>> sourced" and allow the transition?
>>
>>
>> In v2 you can create an image and then add a location for it (this
>> should be enabled, of course). You can even use cinder locations since
>> Kilo (or Havana?).
>
> I don't believe Cinder backing store / Cinder locations can work for
> the Nova use case here.  In the case of this use case, Nova snapshot
> of Cinder backed VM, Nova creates Cinder snapshots of every attached
> Cinder volume and puts the snapshots in a BDM list in the Glance
> metadata, and moves the image to Active with size = 0 and no volume.
>
> This inherently creates an image that is pointing at a collection of
> multiple image binaries (Cinder snapshots) that collectively make up
> the "image".  For example the Cinder snapshots could be snapshot0 =
> boot disk, snapshot1 = application binaries, snapshot1 = DB volume 1
> snapshot2 = DB volume 2.
>
> I may be wrong, but my understanding of multiple location support in
> Glance is that the multiple locations are intended to all contain
> copies / mirrors of the same image binary (singular), and the image
> consumer, say, a Nova virt driver, could look at the locations and
> make the best choice of location based on its settings for ephemeral
> disk backing, etc.  If this is the correct understanding then the
> multiple locations support using Cinder the Cinder backing store to
> reference multiple Cinder snapshots of different Cinder volumes is not
> a good fit.
>
> Now possibly a compromise could exist where in the "snapshot volume
> backed VM" flow, Nova changes to set location of the Glance image to
> be the Cinder snapshot of the boot disk / root BDM, and then puts the
> non-root BDM volumes in the block_device_mapping image metadata.
>
> This would make the image non-empty and would use the Cinder backing
> store for the boot disk while maintaining the existing functionality
> of having the image refer to the other Cinder snapshots in the block
> device mapping metadata.

I sent the previous reply a bit prematurely.  In order for my idea
above to work, to have the Glance image use a Cinder location /
backing store for the boot disk in this case, the Cinder backing store
[1] needs to be enhanced to support Cinder snapshots as well as Cinder
volumes.
[1] https://github.com/openstack/glance_store/blob/master/glance_store/_drivers/cinder.py

>
>
>>
>> Cheers,
>> Flavio
>>
>> --
>> @flaper87
>> Flavio Percoco
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>