Open Stack

Sam, Flavio, Zhenyu, Daniel thank you for your responses!

Indeed, we're in a difficult position with activating zero-size images...
To fix it we have to solve 2 problems: one with disk and container formats
parameters - glance v2 doesn't allow activate images if these two are
unset; and the second with image data - glance v2 requires some data to be
passed before image activation.

About formats: I don't see any good solution rather than to hardcode them
for Glance and then ignore in Nova. If you can figure out a better one,
please share :) But changing image schema - no way!

About data: afaiu, block_device_mapping property contains volume_id. That
means we may take it out and activate an image by providing new location,
i.e. cinder://volume_id. The problem is setting custom locations for image
is disabled by default and enabling it may cause serious mishaps.
My solution was to provide empty data (not 1 byte as Sam said) with
image-upload, it leads to image activation and it works. Unfortunately,
it's just another workaround, because we create absolutely unnecessary
empty file in store and after that users can download it, which is wrong.
But we can improve it on glance side: if we provide empty data in
image-upload, glance won't create a file in store and set location to the
image (in other words - avoid Location layer in Glance domain model). In
that case these operation will be identical (v1:image-create(size=0) and
v2:image-create()+image-upload(data='')).

Frankly speaking I wish to see more feedback on this matter, because we
will have to make difficult decisions or Nova will never adopt Glance v2
api.

On Fri, Jan 8, 2016 at 10:14 PM, Sam Matzek <matzeksam at gmail.com> wrote:

> On Fri, Jan 8, 2016 at 11:54 AM, Sam Matzek <matzeksam at gmail.com> wrote:
> > On Fri, Jan 8, 2016 at 8:31 AM, Flavio Percoco <flavio at redhat.com>
> wrote:
> >> On 29/12/15 07:41 -0600, Sam Matzek wrote:
> >>>
> >>> On Thu, Dec 24, 2015 at 7:49 AM, Mikhail Fedosin <
> mfedosin at mirantis.com>
> >>> wrote:
> >>>>
> >>>> Hello, it's another topic about glance v2 adoption in Nova, but it's
> >>>> different from the others. I want to declare that there is a set of
> >>>> commits,
> >>>> that make Nova version agnostic and allow to work with both glance
> apis.
> >>>> The
> >>>> idea of the solution is to determine the current api version in the
> >>>> beginning and make appropriate requests after.
> >>>> (https://review.openstack.org/#/c/228578/,
> >>>> https://review.openstack.org/#/c/238309/,
> >>>> https://review.openstack.org/#/c/259097/)
> >>>>
> >>>> Indeed, it requires some additional (and painful) work, but now all
> >>>> tempest
> >>>> tests pass in Jenkins.
> >>>>
> >>>> Note: this thread is not about xenplugin, there is another topic,
> called
> >>>> 'Xenplugin + Glance_v2 = Hate'
> >>>>
> >>>> Here the main issues we faced and how we've solved them:
> >>>>
> >>>> 1. "changes-since" filter for image-list is not supported in v2 api.
> >>>> Steve
> >>>> Lewis made a great job and implemented a set of filters with
> comparison
> >>>> operators https://review.openstack.org/#/c/197388/. Filtering by
> >>>> 'changes-since' is completely similar to 'gte:updated_at'.
> >>>>
> >>>> 2. Filtering by custom properties must have prefix 'property-'. In v2
> >>>> it's
> >>>> not required.
> >>>>
> >>>> 3. V2 states that all custom properties must be image attributes, but
> >>>> Nova
> >>>> assumes that they are included in 'properties' dictionary. It's fixed
> >>>> with
> >>>> glanceclient's method 'is_base_property(prop_name)', that returns
> False
> >>>> in
> >>>> case of custom property.
> >>>>
> >>>> 4. is_public=True/False is visibility="public"/"private" respectively.
> >>>>
> >>>> 5. Deleting custom image properties in Nova is performed with
> >>>> 'purge_props'
> >>>> flag. If it's set to True, then all prop names, that are not included
> in
> >>>> updated data, will be removed. In case of v2 we have to explicitly
> >>>> specify
> >>>> prop names in the list param 'remove_props'. To implement this
> behaviour,
> >>>> if
> >>>> 'purge_props' is set, we make additional 'get' request to determine
> the
> >>>> existing properties and put in 'remove_prop' list only those, that are
> >>>> not
> >>>> in updated_data.
> >>>>
> >>>> 6. My favourite:
> >>>> There is an ability to activate an empty image by just providing
> 'size =
> >>>> 0':
> >>>> https://review.openstack.org/#/c/9715/, in this case image will be a
> >>>> collection of metadata. Glance v2 doesn't support this "feature" and
> >>>> that's
> >>>> why we have to implement a very dirty workaround:
> >>>>     * v2 requires, that disk_format and container-format must be set
> >>>> before
> >>>> the activation. if these params are not provided to 'create' method
> then
> >>>> we
> >>>> hardcode it to 'qcow2' and 'bare'.
> >>>>     * we call 'upload' method with empty data (data = '') to activate
> >>>> image.
> >>>> Me (and the rest glance team) think that this image activation with
> >>>> zero-size is inconsistent and we won't implement it in v2. So, I'm
> going
> >>>> to
> >>>> ask if Nova really needs this feature and maybe it's possible to make
> it
> >>>> work without it.
> >>>
> >>>
> >>> Nova uses this functionality when it creates snapshots of volume
> >>> backed instances, that is, instances that only have Cinder volumes
> >>> attached and do not have an ephemeral disk.
> >>> In this case Nova API creates Cinder snapshots for the Cinder volumes
> >>> and builds the block_device_mapping property with block devices that
> >>> reference the Cinder snapshots.  Nova activates this image with size=0
> >>> because this image does not have a disk and simply refers to the
> >>> collection of Cinder snapshots that collectively comprise the binary
> >>> image.  Callers of Glance outside of Nova may also use the APIs to
> >>> create "block device mapping" images as well that contain references
> >>> to Cinder volumes to attach, blank volumes to create, snapshots to
> >>> create volumes from, etc during the server creation.  Not having the
> >>> ability to create these images with V2 is a loss of function.
> >>
> >>
> >> I disagree. Being able to activate empty images breaks the consistency
> >> of Glances API and I don't think it should be considered a feature but
> >> a bug. An active image is an image that can be used to *boot* a VM. If
> >> the image is empty, you simply can't do that.
> >>
> >> Note that pointing to a block device makes the image "non-empty"
> >> already. There's a spec merged[0] and code written[1] to allow for
> >> using cinder as backend for Glance.
> >>
> >> If there's a missing functionality for the mapping users require, I
> >> believe activating empty images is not the solution for it.
> >>
> >> [0] https://review.openstack.org/183363
> >> [1] https://review.openstack.org/166414
> >>
> >>> The callers could supply 1 byte of junk data, like a space character,
> >>> but that would result in a junk image being stored in Glance's default
> >>> backing store for the image.  It would also give the impression that a
> >>> real disk image exists for the image in a backing store which could be
> >>> fetched, which is incorrect.
> >>>
> >>> If I remember correctly Glance V2 lets you transition an image from
> >>> queued to active state with size = 0 if the image is using an external
> >>> backing store such as http.  The store is then called to fetch the
> >>> size.  Would it be possible to allow Glance to consider images with
> >>> size = 0 and the block_device_mapping property to be "externally
> >>> sourced" and allow the transition?
> >>
> >>
> >> In v2 you can create an image and then add a location for it (this
> >> should be enabled, of course). You can even use cinder locations since
> >> Kilo (or Havana?).
> >
> > I don't believe Cinder backing store / Cinder locations can work for
> > the Nova use case here.  In the case of this use case, Nova snapshot
> > of Cinder backed VM, Nova creates Cinder snapshots of every attached
> > Cinder volume and puts the snapshots in a BDM list in the Glance
> > metadata, and moves the image to Active with size = 0 and no volume.
> >
> > This inherently creates an image that is pointing at a collection of
> > multiple image binaries (Cinder snapshots) that collectively make up
> > the "image".  For example the Cinder snapshots could be snapshot0 =
> > boot disk, snapshot1 = application binaries, snapshot1 = DB volume 1
> > snapshot2 = DB volume 2.
> >
> > I may be wrong, but my understanding of multiple location support in
> > Glance is that the multiple locations are intended to all contain
> > copies / mirrors of the same image binary (singular), and the image
> > consumer, say, a Nova virt driver, could look at the locations and
> > make the best choice of location based on its settings for ephemeral
> > disk backing, etc.  If this is the correct understanding then the
> > multiple locations support using Cinder the Cinder backing store to
> > reference multiple Cinder snapshots of different Cinder volumes is not
> > a good fit.
> >
> > Now possibly a compromise could exist where in the "snapshot volume
> > backed VM" flow, Nova changes to set location of the Glance image to
> > be the Cinder snapshot of the boot disk / root BDM, and then puts the
> > non-root BDM volumes in the block_device_mapping image metadata.
> >
> > This would make the image non-empty and would use the Cinder backing
> > store for the boot disk while maintaining the existing functionality
> > of having the image refer to the other Cinder snapshots in the block
> > device mapping metadata.
>
> I sent the previous reply a bit prematurely.  In order for my idea
> above to work, to have the Glance image use a Cinder location /
> backing store for the boot disk in this case, the Cinder backing store
> [1] needs to be enhanced to support Cinder snapshots as well as Cinder
> volumes.
> [1]
> https://github.com/openstack/glance_store/blob/master/glance_store/_drivers/cinder.py
>
> >
> >
> >>
> >> Cheers,
> >> Flavio
> >>
> >> --
> >> @flaper87
> >> Flavio Percoco
> >>
> >>
> __________________________________________________________________________
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160111/2f0e39d5/attachment.html>