Open Stack

On 11/16/2016 09:34 AM, Monty Taylor wrote:
> On 11/15/2016 11:26 PM, joehuang wrote:
>>> Glance Image Uploads and Swift Object Uploads (and downloads). Having
>>> those two data operations go through an API proxy seems inefficient.
>>> However, having them not in the API seems like a bad user experience.
>>> Perhaps if we take advantage of the gRPC streaming protocol support
>>> doing a direct streaming passthrough actually wouldn't be awful. Or
>>> maybe the better approach would be for the gRPC call to return a URL and
>>> token for a user to POST/PUT to directly. Literally no clue.
>>
>> From bandwidth consideration, the bandwidth for the API service like Oaktree 
>> may not as wide as that for data storage service, for example Swift. That means
>> if the Oaktree will proxy the image upload, then the bandwidth for the Oaktree
>> sever may be exhausted soon, and not  able to provide other API service.
> 
> Yes - this is exactly right and a big part of the problem.
> 
>> It's good in Glance V2 that image could be update to a store, then register the location
>> to a Glance image, but not directly upload bits to Glance API directly.
> 
> Unfortunately for us - we need to support glance v1 PUT, glance v2 PUT,
> glance v2 task import and the new and upcoming glance v2 multi-step
> image upload.
> 
> I had an idea this morning though - tell me what you think.
> 
> The API will be multi-step (similar to the new glance image upload
> process) but with explicit instructions for users. We'll suggest that
> client lib authors who are building friendly libs on top of the oaktree
> client encapsulate the multi-step logic in some manner, and we will
> provide explicit instructions on what the multi-steps are.
> 
> API:
> 
> rpc CreateImage (ImageSpec) returns (ImageUpload) {}
> rpc UploadImageContent (stream ImageContent) returns (ImageUploadStatus) {}
> rpc FinalizeImageUpload (ImageSpec) returns (Image) {}
> 
> rpc GetToken (Location) returns (Token) {}
> 
> message ImageSpec {
>   Location location = 1;
>   string name = 3;
>   uint32 min_ram = 4;
>   uint64 min_disk = 5;
>   // etc - more fields
>   repeated bytes image_content = 99;
> };
> 
> message ImageUpload {
>   enum UploadScheme {
>     grpc_upload = 0;
>     rest_put = 1;
>     swift = 2;
>   };
>   UploadScheme scheme = 1;
>   string endpoint = 2;

Ooh! What if endpoint was actually a repeated field (array)? That way
for PUT operations it would just be a single entry - but for the swift
case, the SLO segment URLs could be pre-computed by oaktree.

It would make "size" a hard requirement from the API - but I'm fine with
that.

Logic below ...

>   map<string, string> headers = 3;
>   uint32 segement_size = 4;
> };
> 
> The logic is then:
> 
> image_spec = ImageSpec(
>     name='my_image')
> upload = client.CreateImage(image_spec)
> if upload.scheme == ImageUpload.grpc_upload:
>     image = client.UploadImage(open('file', 'r'))
> elif upload.scheme == ImageUpload.rest_put:
>     image = requests.put(
>         upload.endpoint, headers=upload.headers,
>         data=open('file', 'r'))
> elif upload.scheme = ImageUpload.swift:
>     # upload to upload.endpoint, probably as a
>     # swift SLO splitting the content into
>     # segments of upload.segment_size
     count = 0
     content = open('file', 'r')
     for endpoint in upload.endpoints:
         content.seek(count * upload.segment_size)
         requests.put(
             endpoint, headers=upload.headers,
             data=content.read(upload.segment_size))
         count += 1

Making that multi-threaded is an obvious improvement of course.

> image = client.FinalizeImageUpload(image_spec)

Then the creation of the manifest object in swift could be handled in
finalize by oaktree. In fact- that way we could collapse the put and
swift cases to just be a "REST" case - since all of the operations are
PUT to a URL provided by oaktree - and for glance PUT segment_size will
just be == size.

> It's a three-pronged upload approach that a client author has to write -
> but the two different REST interactions should be easy - the grpc
> endpoint should be able to return endpoint/headers/token so that the end
> user doens't have to interpret _What_ rest call to make - just needs to
> make the exact one that ImageUpload message describes. (the swift upload
> can be documented more precisely - but this email is already long)
> 
> For the swift case, it's possible that the token could expire before all
> of the PUTs are made for each of the image segments. That's why we add a
> GetToken api call - so that in a loop the client can just request
> another token from the gRPC api without having to know anything more
> about those mechanics. Obviously that can also be hidden by client libs
> too - but in a way that's easily replicatable across langauges - and if
> someone wants to do things by hand, there are very explicit instructions.
> 
> The finalize step is important because there are things that may need to
> be performed after the upload in all cases. For the swift case, the
> import task has to be spawned and waited on. For the put and gRPC cases
> there are metadata fields, like protected, that can only be set once the
> other actions are complete.
> 
> (there are parts of this that are hand-wavey - but how does it sound in
> general?)
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>