Open Stack

Hey all,

I'm well aware that this isn't exactly timely because we've partially agreed on the import spec in large strokes. [1]_ I haven't had a great deal of time before now to really dig into the spec, though, so I'm bringing my feedback. I've spoken to some API-WG members and other Glance team members and I thought it would be best to discuss some of the concerns here.

An outline (or tl;dr) is roughly

- We're exposing way too much complexity to the user
- The user has to be very aware of discovering all of the data from the cloud
  - The user has to store a lot of that data to make subsequent requests
  - Exposing information to the user is not bad, making the user resubmit it to the cloud is bad
- We're introducing two new image states. Do we need two?
- Quotas - Are these even still a thing?


So the problem description explains why uploading an image can be very complex and why some clouds need to not allow uploads directly to Glance. I understand that very well. I also recognize that I am the person who advocated so strongly for having information endpoints that help the user discover information about the cloud. I think this allows for us to make smarter clients that can help the user do the right thing (DTRT) instead of shooting themselves in the foot accidentally.

All of that said, I have some very severe reservations about the extra surface area we're adding to the images API and in particular how a user's view of the image upload flow would work. We've shipped v2 of Glance as the default version now in python-glanceclient (as of Liberty) so users are already going to be familiar with a two-step dance of creating an image:

1. Create the image record (image-create)
2. Upload the image data (image-upload)

In this new workflow, the dance is extended

1. Get information about how the cloud is configured and prefers images
2. Create the image record
3. Upload the image data somewhere (directly to Glance, or to some other service in that cloud perhaps [e.g., swift]) that is specified in the data retrieved from step 1
4. Tell Glance to import/process the image

Here are my issues with this new upload workflow:

1. The user must pre-fetch the information to have any chance of their upload being successful
2. They have to use that information to find out where to upload the image bytes (data) to
3. In step 4, they have to tell Glance if they're using the direct/indirect (currently named glance-direct and swift-local respectively) and then give it information (retrieved from step 1) about what the target formats are for that cloud

Here's the thing, the desire to have data hosted elsewhere that is uploaded directly there is not a new desire in the cloud, this community, or in general. We know that people have been asking for a glance-store refactor so other services can read/write data directly from wherever it is stored. (This may seem a bit orthogonal but follow along with me for a second.)

We also already know that Amazon expects image data uploads to be pushed to S3 first and then imported. So our workflow wouldn't be abnormal (except that we don't want to force all OpenStack clouds to have Swift or a storage service that it uses).

We can also look at how a service like GitHub handles arbitrary data uploads from users via their API:
- https://developer.github.com/v3/repos/releases/#create-a-release
- https://developer.github.com/v3/repos/releases/#upload-a-release-asset

Quoting for the purpose of reducing how much you have to read:

> This endpoint makes use of a Hypermedia relation to determine which URL to access. This endpoint
> is provided by a URI template in the release's API response. 
> [snip]
> The asset data is expected in its raw binary form, rather than JSON. Everything else about the
> endpoint is the same as the rest of the API. For example, you'll still need to pass your authentication
> to be able to upload an asset.

In other words, when you create a Release, it has associated Assets which you upload directly to (in this case S3). You're told where to upload it to when you create the Release (also any time you retrieve information about it).

So what if we modeled this a little

1. The user creates an image record (image-create)
2. The user uses the link in the response to upload the image data (we don't care whether it is direct or not because the user can follow it without thinking about it)
3. The user sends a request to process the image without having to specify what to translate the image to because the operator has chosen a preferred image type (and maybe has other acceptable types that mean the image will not need to be converted if it's already of that type).

This adds an extra step and it could remove the need for an "uploading" status.

This also means that the user does not need to be cognizant of the method that they're using to create an image (glance-direct, swift-local, foo-bar-bogus-whatever, etc.) and that they do not need to think too hard about the target format for that cloud or pre-process any extra information or worry whether that cloud needs some location URL for an indirect upload. Glance can manage that complexity for the user.

Now let's talk about step 2 briefly. If the operator wants all image bytes to go to Glance directly, this can be the default behaviour. In that case, the URL would be "https://<glance>/v2/images/<image-id>/files" and the user would do it like the default PUT behaviour that we have now. If it's an indirect upload, then the user would make another PUT request with that URL and their authentication credentials. If the user is out of storage quota than that service will tell the user.

I think in this way, we can provide the user some information about their uploaded data as well as well as letting admins handle quota. This still doesn't discuss giving admin more visibility (in a direct upload case) about quota of uploaded but not yet processed data. That's an important question to have, but I don't have an easy answer for it though. I hope someone else does.


References:

[1] https://specs.openstack.org/openstack/glance-specs/specs/mitaka/approved/image-import/image-import-refactor.html

--  
Ian Cordasco