Open Stack

On 13/02/15 17:01 +0100, Jordan Pittier wrote:
>Humm this doesn't have to be complicated, for a start.
>

Sorry for my late reply

>>- Figuring out the http method the server expects (POST/PUT)
>Yeah, I agree. There"s no definitive answer to this but I think PUT makes sense
>here. I googled 'post vs put' and I found that the "idempotent" and "who is in
>charge of the actual resource location choice" (the client vs the server),
>favors PUT. 

Right but that's not what the remote server may be expecting. One of
the problems about the HTTP store is that there's not real "API"
besides what the HTTP protocol allows you to do. That is to say that a
remote server may accept POST/PUT and in order to keep the
implementation non-opinionated, you'd need to have a way for these
things to be specified.

>
>>- Adding support for at least few HTTP auth methods
>Why should the "write" path be more secured/more flexible than the read path ?
>If I take a look at the current HTTP store, only basic auth is supported (ie
>http://user:pass@server1/myLinuxImage). I suggest the write path (ie the add()
>method) should support the same auth mecanism. The cloud admin could also add
>some firewall rules to make sure the HTTP backend server can only be accessed
>by the Glance-api servers.

I didn't say the read path was correct :P

That said, I agree that we should keep both paths consistent.

>>- Having a sufixed URL where we're sure glance will have proper
>> permissions to upload data.
>That's up the the cloud admin/operator to make it work. The HTTP glance_store
>could have 2 config flags :
>a) "http_server", a string with the scheme (http vs https) and the hostname of
>the HTTP server, ie 'http://server1' 
>b) "path_prefix". A string that will prefix the "path" part of the image URL.
>This config flag could be left empty/is optional. 

Yes, it was probably not clear from my previous email that these were
not "ands" but things that would need to be brought up.

>
>>Handling HTTP responses from the server
>That's of course to be discussed. But, IMO, this should be as simple as "if
>response.code is 200 or 202 then OKAY else raise GlanceStoreException". I am
>not sure any other glance store is more "granular" than this. 

Again, this assumes "too much" from the server. So, either we impose
some kind of rules as to how Glance expects the HTTP server to behave
or we try to be bullet proof API-wise.

>>How can we handle quota?
>I am new to glance_store, is there a notion of quotas in glance stores ? I
>though Glance (API) was handling this. What kind of quotas are we talking about
>here ?

Glance handles quotas. The problem is that when the data is sent to
the remote store, glance looses some control on it. A user may upload
some data, the HTTP push could fail and we may try to delete the data
without any proof that it will be correctly deleted.

Also, without auth, we will have to force the user to send all image
data through glance. The reason is that we don't know whether the HTTP
store has support for HEAD to report the image size when using
`--location`.

Sorry if all the above sounds confusing. The problem with the HTTP
store is that we have basically no control over it and that is
worrisome from a security and implementation perspective.

Flavio

>Frankly, it shouldn't add that much code. I feel we can make it clean if we
>leverage the different Python modules (httplib etc.) 
>
>Regards,
>Jordan
>
>
>On Fri, Feb 13, 2015 at 4:20 PM, Flavio Percoco <flavio at redhat.com> wrote:
>
>    On 13/02/15 16:01 +0100, Jordan Pittier wrote:
>
>            What is the difference between just calling the Glance API to
>            upload an image,
>
>        versus adding add() functionality to the HTTP image store?
>        You mean using "glance image-create --location http://server1/
>        myLinuxImage [..]
>        " ? If so, I guess adding the add() functionality will save the user
>        from
>        having to find the right POST curl/wget command to properly upload his
>        image.
>
>
>    I believe it's more complex than this. Having an `add` method for the
>    HTTP store implies:
>
>    - Figuring out the http method the server expects (POST/PUT)
>    - Adding support for at least few HTTP auth methods
>    - Having a sufixed URL where we're sure glance will have proper
>     permissions to upload data.
>    - Handling HTTP responses from the server w.r.t the status of the data
>     upload. For example: What happens if the remote http server runs out
>     of space? What's the response status going to be like? How can we
>     make glance agnostic to these discrepancies across HTTP servers so
>     that it's consistent in its responses to glance users?
>    - How can we handle quota?
>
>    I'm not fully opposed, although it sounds like not worth it code-wise,
>    maintenance-wise and performance-wise. The user will have to run just
>    1 command but at the cost of all of the above.
>
>    Do the points listed above make sense to you?
>
>    Cheers,
>    Flavio
>
>
>
>
>        On Fri, Feb 13, 2015 at 3:55 PM, Jay Pipes <jaypipes at gmail.com> wrote:
>
>           On 02/13/2015 09:47 AM, Jordan Pittier wrote:
>                 Hi list,
>
>               I would like to add the 'add' capability to the HTTP glance
>        store.
>
>               Let's say I (as an operator or cloud admin) provide an HTTP
>        server
>               where
>               (authenticated/trusted) users/clients can make the following
>        HTTP
>               request :
>
>               POST http://server1/myLinuxImage HTTP/1.1
>               Host: server1
>               Content-Length: 256000000
>               Content-Type: application/octet-stream
>
>               mybinarydata[..]
>
>               Then the HTTP server will store the binary data, somewhere (for
>               instance
>               locally), some how (for instance in a plain file), so that the
>        data is
>               later on accessible by a simple GET http://server1/myLinuxImage
>
>               In that case, this HTTP server could easily be a full fleshed
>        Glance
>               store.
>
>               Questions :
>               1) Has this been already discussed/proposed ? If so, could
>        someone give
>               me a pointer to this work ?
>               2) Can I start working on this ? (the 2 main work items are :
>        'add an
>               add method to glance_store._drivers.http.__Store' and 'add a
>        delete
>               method to glance_store._drivers.http.__Store (HTTP DELETE
>        method)'
>
>
>           What is the difference between just calling the Glance API to upload
>        an
>           image, versus adding add() functionality to the HTTP image store?
>
>           Best,
>           -jay
>
>         
>         __________________________________________________________________________
>           OpenStack Development Mailing List (not for usage questions)
>           Unsubscribe: OpenStack-dev-request at lists.openstack.org?
>        subject:unsubscribe
>           http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
>        __________________________________________________________________________
>        OpenStack Development Mailing List (not for usage questions)
>        Unsubscribe: OpenStack-dev-request at lists.openstack.org?
>        subject:unsubscribe
>        http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>    --
>    @flaper87
>    Flavio Percoco
>   
>    __________________________________________________________________________
>    OpenStack Development Mailing List (not for usage questions)
>    Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>    http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>

>__________________________________________________________________________
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


-- 
@flaper87
Flavio Percoco
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150217/0760a418/attachment.pgp>