Open Stack

Hi Mark, comments inline. Thanks!

On 05/22/2013 05:57 PM, Mark Washenberger wrote:
> On Wed, May 22, 2013 at 1:50 PM, Jay Pipes <jaypipes at gmail.com
<snip>
> 
>     I don't really see the need to add a whole new workflow and API resource
>     for this.
> 
>     Why not just add the new status called "Verifying" and then place images
>     that are done with the "Saving" state into this "Verifying" state until
>     someone (or something) calls:
> 
>     PUT /images/<UUID>
> 
>     with a body of:
> 
>     {
>      'status': 'ACTIVE'
>     }
> 
>     You can use the existing policy stuff to enforce that only users with
>     certain privileges can change an image's status from Verifying to
>     Active, and images never show up in a user's list of images (public or
>     otherwise) until they are in ACTIVE state anyway.
> 
>     For images added to Glance with copy_from or location, simply change the
>     current behaviour of setting the image status to Active after the image
>     has been copy_from'd instead to Verifying.
> 
>     This way, there's no need to introduce a new /image-imports resource at
>     all, and you don't change the image retrieval workflow at all.
> 
>     Best,
>     -jay
> 
> 
> Hi Jay,
> 
> You bring up good points. In some of the design discussion we considered
> something similar to what you proposed as well. The intentions behind
> making imports a separate action is basically that the image doesn't
> really exist until the import is completed. There are a couple of
> reasons why this "feels right" to me.
> 
> 1) Before import is done, you really can't use the image for anything. 

Same with images that are in the process of being uploaded... there's no
substantive difference between an "image import" and simply uploading an
image.

> 2) During the import process, the image may take on additional metadata,
> either at a deployers discretion with verification, or based on an
> examination of metadata associated with the import source. (I'm thinking
> of the use cases
> from https://blueprints.launchpad.net/glance/+spec/clone-image-across-regions
> and https://blueprints.launchpad.net/glance/+spec/iso-image-metadata).
> So the image is still very much in flux.

This still doesn't mean a whole new REST resource is necessary or that
the workflow is any different from the current image upload flow.

> 3) You probably need to specify a checksum for the data you're uploading
> to an import job, but that checksum is unlikely to match the one that is
> set on the image when it is done importing. That breaks a certain
> put/get parity in an icky way. The same goes for image size, disk and
> container format, and other relevant properties.

I don't understand this. The verification process doesn't change the
bits of the image, so why would the checksum change?

> 4) During the import/verify, there will likely be a lot of failures. It
> would be better if we have a good way to explain what went wrong with
> the failure, but I'd prefer not to add a generic "image_faults"
> resource. It is better IMO to track the fault directly along with an
> entity that represents the action that failed. I imagine it will be
> difficult to find a place to put fault messages inside current image
> metadata.

Now this, I totally understand. That said, I don't believe the proposed
new top-level /image-imports resource is the appropriate way to deal
with this. I could, on the other hand, get behind a proposal to, say,
add a new /images/<UUID>/faults subcollection resource that could be
queried for an image.

> 5) By having import jobs as a separate resource, we never have to bother
> with questions like "Can I delete/modify/share an image during
> import/verify/error?"

I don't buy this argument :) It's as simple as adding a new image status
(Verifying) and changing the couple lines of code that currently handle
state change between image statuses...

> 6) For me, I'd rather not see proper images and image data that is
> currently being imported (or recently failed to be imported) in the same
> list. Though I expect others would have different preferences.

But we already have this... if an image is currently uploading, its
status is "Saving". Users won't see these images in their GET /images
responses until the image gets to Active status. It's the same concept
for the Verifying status here.

> Regarding image retrieval, I think there is a genuine difference between
> export and download. The idea behind download is "give me the bits
> exactly as you have them", whereas with export, the idea is more like
> "save the image over there in a storage area I can control (say, my
> swift container) in a canonical form I can use in other clouds." So
> interoperability stories in the future are more geared towards imports
> and exports, and not uploads and downloads. At least, that's where we
> were leaning after the recent design summit talks about image
> interoperability (IIRC).

Perhaps I'm just not understanding this because I missed all the Glance
sessions at the summit :( But the above kinda sounds like Glance
development is heading towards shattering the up-to-now-set-in-stone
dictate that once an image is uploaded and active, it is immutable.

Has that dictate changed with this new import/export stuff?

Best,
-jay