Open Stack

Jay, Flavio, thanks for this interesting discussion. I get your points and
they really make sense to me.

I'll go for a specific driver that will inherits from the HTTP Store for
the read path and implements the write path.

Jordan

On Tue, Feb 17, 2015 at 12:52 PM, Flavio Percoco <flavio at redhat.com> wrote:

> On 13/02/15 17:01 +0100, Jordan Pittier wrote:
>
>> Humm this doesn't have to be complicated, for a start.
>>
>>
> Sorry for my late reply
>
>  - Figuring out the http method the server expects (POST/PUT)
>>>
>> Yeah, I agree. There"s no definitive answer to this but I think PUT makes
>> sense
>> here. I googled 'post vs put' and I found that the "idempotent" and "who
>> is in
>> charge of the actual resource location choice" (the client vs the server),
>> favors PUT.
>>
>
> Right but that's not what the remote server may be expecting. One of
> the problems about the HTTP store is that there's not real "API"
> besides what the HTTP protocol allows you to do. That is to say that a
> remote server may accept POST/PUT and in order to keep the
> implementation non-opinionated, you'd need to have a way for these
> things to be specified.
>
>
>>  - Adding support for at least few HTTP auth methods
>>>
>> Why should the "write" path be more secured/more flexible than the read
>> path ?
>> If I take a look at the current HTTP store, only basic auth is supported
>> (ie
>> http://user:pass@server1/myLinuxImage). I suggest the write path (ie the
>> add()
>> method) should support the same auth mecanism. The cloud admin could also
>> add
>> some firewall rules to make sure the HTTP backend server can only be
>> accessed
>> by the Glance-api servers.
>>
>
> I didn't say the read path was correct :P
>
> That said, I agree that we should keep both paths consistent.
>
>  - Having a sufixed URL where we're sure glance will have proper
>>> permissions to upload data.
>>>
>> That's up the the cloud admin/operator to make it work. The HTTP
>> glance_store
>> could have 2 config flags :
>> a) "http_server", a string with the scheme (http vs https) and the
>> hostname of
>> the HTTP server, ie 'http://server1'
>> b) "path_prefix". A string that will prefix the "path" part of the image
>> URL.
>> This config flag could be left empty/is optional.
>>
>
> Yes, it was probably not clear from my previous email that these were
> not "ands" but things that would need to be brought up.
>
>
>>  Handling HTTP responses from the server
>>>
>> That's of course to be discussed. But, IMO, this should be as simple as
>> "if
>> response.code is 200 or 202 then OKAY else raise GlanceStoreException". I
>> am
>> not sure any other glance store is more "granular" than this.
>>
>
> Again, this assumes "too much" from the server. So, either we impose
> some kind of rules as to how Glance expects the HTTP server to behave
> or we try to be bullet proof API-wise.
>
>  How can we handle quota?
>>>
>> I am new to glance_store, is there a notion of quotas in glance stores ? I
>> though Glance (API) was handling this. What kind of quotas are we talking
>> about
>> here ?
>>
>
> Glance handles quotas. The problem is that when the data is sent to
> the remote store, glance looses some control on it. A user may upload
> some data, the HTTP push could fail and we may try to delete the data
> without any proof that it will be correctly deleted.
>
> Also, without auth, we will have to force the user to send all image
> data through glance. The reason is that we don't know whether the HTTP
> store has support for HEAD to report the image size when using
> `--location`.
>
> Sorry if all the above sounds confusing. The problem with the HTTP
> store is that we have basically no control over it and that is
> worrisome from a security and implementation perspective.
>
> Flavio
>
>
>  Frankly, it shouldn't add that much code. I feel we can make it clean if
>> we
>> leverage the different Python modules (httplib etc.)
>>
>> Regards,
>> Jordan
>>
>>
>> On Fri, Feb 13, 2015 at 4:20 PM, Flavio Percoco <flavio at redhat.com>
>> wrote:
>>
>>    On 13/02/15 16:01 +0100, Jordan Pittier wrote:
>>
>>            What is the difference between just calling the Glance API to
>>            upload an image,
>>
>>        versus adding add() functionality to the HTTP image store?
>>        You mean using "glance image-create --location http://server1/
>>        myLinuxImage [..]
>>        " ? If so, I guess adding the add() functionality will save the
>> user
>>        from
>>        having to find the right POST curl/wget command to properly upload
>> his
>>        image.
>>
>>
>>    I believe it's more complex than this. Having an `add` method for the
>>    HTTP store implies:
>>
>>    - Figuring out the http method the server expects (POST/PUT)
>>    - Adding support for at least few HTTP auth methods
>>    - Having a sufixed URL where we're sure glance will have proper
>>     permissions to upload data.
>>    - Handling HTTP responses from the server w.r.t the status of the data
>>     upload. For example: What happens if the remote http server runs out
>>     of space? What's the response status going to be like? How can we
>>     make glance agnostic to these discrepancies across HTTP servers so
>>     that it's consistent in its responses to glance users?
>>    - How can we handle quota?
>>
>>    I'm not fully opposed, although it sounds like not worth it code-wise,
>>    maintenance-wise and performance-wise. The user will have to run just
>>    1 command but at the cost of all of the above.
>>
>>    Do the points listed above make sense to you?
>>
>>    Cheers,
>>    Flavio
>>
>>
>>
>>
>>        On Fri, Feb 13, 2015 at 3:55 PM, Jay Pipes <jaypipes at gmail.com>
>> wrote:
>>
>>           On 02/13/2015 09:47 AM, Jordan Pittier wrote:
>>                 Hi list,
>>
>>               I would like to add the 'add' capability to the HTTP glance
>>        store.
>>
>>               Let's say I (as an operator or cloud admin) provide an HTTP
>>        server
>>               where
>>               (authenticated/trusted) users/clients can make the following
>>        HTTP
>>               request :
>>
>>               POST http://server1/myLinuxImage HTTP/1.1
>>               Host: server1
>>               Content-Length: 256000000
>>               Content-Type: application/octet-stream
>>
>>               mybinarydata[..]
>>
>>               Then the HTTP server will store the binary data, somewhere
>> (for
>>               instance
>>               locally), some how (for instance in a plain file), so that
>> the
>>        data is
>>               later on accessible by a simple GET
>> http://server1/myLinuxImage
>>
>>               In that case, this HTTP server could easily be a full
>> fleshed
>>        Glance
>>               store.
>>
>>               Questions :
>>               1) Has this been already discussed/proposed ? If so, could
>>        someone give
>>               me a pointer to this work ?
>>               2) Can I start working on this ? (the 2 main work items are
>> :
>>        'add an
>>               add method to glance_store._drivers.http.__Store' and 'add
>> a
>>        delete
>>               method to glance_store._drivers.http.__Store (HTTP DELETE
>>        method)'
>>
>>
>>           What is the difference between just calling the Glance API to
>> upload
>>        an
>>           image, versus adding add() functionality to the HTTP image
>> store?
>>
>>           Best,
>>           -jay
>>
>>
>>         ___________________________________________________________
>> _______________
>>           OpenStack Development Mailing List (not for usage questions)
>>           Unsubscribe: OpenStack-dev-request at lists.openstack.org?
>>        subject:unsubscribe
>>           http://lists.openstack.org/cgi-bin/mailman/listinfo/
>> openstack-dev
>>
>>
>>
>>
>>        ____________________________________________________________
>> ______________
>>        OpenStack Development Mailing List (not for usage questions)
>>        Unsubscribe: OpenStack-dev-request at lists.openstack.org?
>>        subject:unsubscribe
>>        http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>>    --
>>    @flaper87
>>    Flavio Percoco
>>      ____________________________________________________________
>> ______________
>>    OpenStack Development Mailing List (not for usage questions)
>>    Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:
>> unsubscribe
>>    http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>>
>  ____________________________________________________________
>> ______________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:
>> unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> --
> @flaper87
> Flavio Percoco
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150217/22361707/attachment.html>