Open Stack

On 11/15/2016 11:26 PM, joehuang wrote:
>> Glance Image Uploads and Swift Object Uploads (and downloads). Having
>> those two data operations go through an API proxy seems inefficient.
>> However, having them not in the API seems like a bad user experience.
>> Perhaps if we take advantage of the gRPC streaming protocol support
>> doing a direct streaming passthrough actually wouldn't be awful. Or
>> maybe the better approach would be for the gRPC call to return a URL and
>> token for a user to POST/PUT to directly. Literally no clue.
> 
> From bandwidth consideration, the bandwidth for the API service like Oaktree 
> may not as wide as that for data storage service, for example Swift. That means
> if the Oaktree will proxy the image upload, then the bandwidth for the Oaktree
> sever may be exhausted soon, and not  able to provide other API service.

Yes - this is exactly right and a big part of the problem.

> It's good in Glance V2 that image could be update to a store, then register the location
> to a Glance image, but not directly upload bits to Glance API directly.

Unfortunately for us - we need to support glance v1 PUT, glance v2 PUT,
glance v2 task import and the new and upcoming glance v2 multi-step
image upload.

I had an idea this morning though - tell me what you think.

The API will be multi-step (similar to the new glance image upload
process) but with explicit instructions for users. We'll suggest that
client lib authors who are building friendly libs on top of the oaktree
client encapsulate the multi-step logic in some manner, and we will
provide explicit instructions on what the multi-steps are.

API:

rpc CreateImage (ImageSpec) returns (ImageUpload) {}
rpc UploadImageContent (stream ImageContent) returns (ImageUploadStatus) {}
rpc FinalizeImageUpload (ImageSpec) returns (Image) {}

rpc GetToken (Location) returns (Token) {}

message ImageSpec {
  Location location = 1;
  string name = 3;
  uint32 min_ram = 4;
  uint64 min_disk = 5;
  // etc - more fields
  repeated bytes image_content = 99;
};

message ImageUpload {
  enum UploadScheme {
    grpc_upload = 0;
    rest_put = 1;
    swift = 2;
  };
  UploadScheme scheme = 1;
  string endpoint = 2;
  map<string, string> headers = 3;
  uint32 segement_size = 4;
};

The logic is then:

image_spec = ImageSpec(
    name='my_image')
upload = client.CreateImage(image_spec)
if upload.scheme == ImageUpload.grpc_upload:
    image = client.UploadImage(open('file', 'r'))
elif upload.scheme == ImageUpload.rest_put:
    image = requests.put(
        upload.endpoint, headers=upload.headers,
        data=open('file', 'r'))
elif upload.scheme = ImageUpload.swift:
    # upload to upload.endpoint, probably as a
    # swift SLO splitting the content into
    # segments of upload.segment_size
image = client.FinalizeImageUpload(image_spec)

It's a three-pronged upload approach that a client author has to write -
but the two different REST interactions should be easy - the grpc
endpoint should be able to return endpoint/headers/token so that the end
user doens't have to interpret _What_ rest call to make - just needs to
make the exact one that ImageUpload message describes. (the swift upload
can be documented more precisely - but this email is already long)

For the swift case, it's possible that the token could expire before all
of the PUTs are made for each of the image segments. That's why we add a
GetToken api call - so that in a loop the client can just request
another token from the gRPC api without having to know anything more
about those mechanics. Obviously that can also be hidden by client libs
too - but in a way that's easily replicatable across langauges - and if
someone wants to do things by hand, there are very explicit instructions.

The finalize step is important because there are things that may need to
be performed after the upload in all cases. For the swift case, the
import task has to be spawned and waited on. For the put and gRPC cases
there are metadata fields, like protected, that can only be set once the
other actions are complete.

(there are parts of this that are hand-wavey - but how does it sound in
general?)