<div dir="ltr">Jay, Flavio, thanks for this interesting discussion. I get your points and they really make sense to me.<div><br></div><div>I'll go for a specific driver that will inherits from the HTTP Store for the read path and implements the write path.</div><div><br></div><div>Jordan</div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Feb 17, 2015 at 12:52 PM, Flavio Percoco <span dir="ltr"><<a href="mailto:flavio@redhat.com" target="_blank">flavio@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span>On 13/02/15 17:01 +0100, Jordan Pittier wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Humm this doesn't have to be complicated, for a start.<br>
<br>
</blockquote>
<br></span>
Sorry for my late reply<span><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
- Figuring out the http method the server expects (POST/PUT)<br>
</blockquote>
Yeah, I agree. There"s no definitive answer to this but I think PUT makes sense<br>
here. I googled 'post vs put' and I found that the "idempotent" and "who is in<br>
charge of the actual resource location choice" (the client vs the server),<br>
favors PUT. <br>
</blockquote>
<br></span>
Right but that's not what the remote server may be expecting. One of<br>
the problems about the HTTP store is that there's not real "API"<br>
besides what the HTTP protocol allows you to do. That is to say that a<br>
remote server may accept POST/PUT and in order to keep the<br>
implementation non-opinionated, you'd need to have a way for these<br>
things to be specified.<span><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
- Adding support for at least few HTTP auth methods<br>
</blockquote>
Why should the "write" path be more secured/more flexible than the read path ?<br>
If I take a look at the current HTTP store, only basic auth is supported (ie<br>
http://user:pass@server1/<u></u>myLinuxImage). I suggest the write path (ie the add()<br>
method) should support the same auth mecanism. The cloud admin could also add<br>
some firewall rules to make sure the HTTP backend server can only be accessed<br>
by the Glance-api servers.<br>
</blockquote>
<br></span>
I didn't say the read path was correct :P<br>
<br>
That said, I agree that we should keep both paths consistent.<span><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
- Having a sufixed URL where we're sure glance will have proper<br>
permissions to upload data.<br>
</blockquote>
That's up the the cloud admin/operator to make it work. The HTTP glance_store<br>
could have 2 config flags :<br>
a) "http_server", a string with the scheme (http vs https) and the hostname of<br>
the HTTP server, ie '<a href="http://server1" target="_blank">http://server1</a>' <br>
b) "path_prefix". A string that will prefix the "path" part of the image URL.<br>
This config flag could be left empty/is optional. <br>
</blockquote>
<br></span>
Yes, it was probably not clear from my previous email that these were<br>
not "ands" but things that would need to be brought up.<span><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Handling HTTP responses from the server<br>
</blockquote>
That's of course to be discussed. But, IMO, this should be as simple as "if<br>
response.code is 200 or 202 then OKAY else raise GlanceStoreException". I am<br>
not sure any other glance store is more "granular" than this. <br>
</blockquote>
<br></span>
Again, this assumes "too much" from the server. So, either we impose<br>
some kind of rules as to how Glance expects the HTTP server to behave<br>
or we try to be bullet proof API-wise.<span><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
How can we handle quota?<br>
</blockquote>
I am new to glance_store, is there a notion of quotas in glance stores ? I<br>
though Glance (API) was handling this. What kind of quotas are we talking about<br>
here ?<br>
</blockquote>
<br></span>
Glance handles quotas. The problem is that when the data is sent to<br>
the remote store, glance looses some control on it. A user may upload<br>
some data, the HTTP push could fail and we may try to delete the data<br>
without any proof that it will be correctly deleted.<br>
<br>
Also, without auth, we will have to force the user to send all image<br>
data through glance. The reason is that we don't know whether the HTTP<br>
store has support for HEAD to report the image size when using<br>
`--location`.<br>
<br>
Sorry if all the above sounds confusing. The problem with the HTTP<br>
store is that we have basically no control over it and that is<br>
worrisome from a security and implementation perspective.<span><font color="#888888"><br>
<br>
Flavio</font></span><div><div><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Frankly, it shouldn't add that much code. I feel we can make it clean if we<br>
leverage the different Python modules (httplib etc.) <br>
<br>
Regards,<br>
Jordan<br>
<br>
<br>
On Fri, Feb 13, 2015 at 4:20 PM, Flavio Percoco <<a href="mailto:flavio@redhat.com" target="_blank">flavio@redhat.com</a>> wrote:<br>
<br>
On 13/02/15 16:01 +0100, Jordan Pittier wrote:<br>
<br>
What is the difference between just calling the Glance API to<br>
upload an image,<br>
<br>
versus adding add() functionality to the HTTP image store?<br>
You mean using "glance image-create --location <a href="http://server1/" target="_blank">http://server1/</a><br>
myLinuxImage [..]<br>
" ? If so, I guess adding the add() functionality will save the user<br>
from<br>
having to find the right POST curl/wget command to properly upload his<br>
image.<br>
<br>
<br>
I believe it's more complex than this. Having an `add` method for the<br>
HTTP store implies:<br>
<br>
- Figuring out the http method the server expects (POST/PUT)<br>
- Adding support for at least few HTTP auth methods<br>
- Having a sufixed URL where we're sure glance will have proper<br>
permissions to upload data.<br>
- Handling HTTP responses from the server w.r.t the status of the data<br>
upload. For example: What happens if the remote http server runs out<br>
of space? What's the response status going to be like? How can we<br>
make glance agnostic to these discrepancies across HTTP servers so<br>
that it's consistent in its responses to glance users?<br>
- How can we handle quota?<br>
<br>
I'm not fully opposed, although it sounds like not worth it code-wise,<br>
maintenance-wise and performance-wise. The user will have to run just<br>
1 command but at the cost of all of the above.<br>
<br>
Do the points listed above make sense to you?<br>
<br>
Cheers,<br>
Flavio<br>
<br>
<br>
<br>
<br>
On Fri, Feb 13, 2015 at 3:55 PM, Jay Pipes <<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>> wrote:<br>
<br>
On 02/13/2015 09:47 AM, Jordan Pittier wrote:<br>
Hi list,<br>
<br>
I would like to add the 'add' capability to the HTTP glance<br>
store.<br>
<br>
Let's say I (as an operator or cloud admin) provide an HTTP<br>
server<br>
where<br>
(authenticated/trusted) users/clients can make the following<br>
HTTP<br>
request :<br>
<br>
POST <a href="http://server1/myLinuxImage" target="_blank">http://server1/myLinuxImage</a> HTTP/1.1<br>
Host: server1<br>
Content-Length: 256000000<br>
Content-Type: application/octet-stream<br>
<br>
mybinarydata[..]<br>
<br>
Then the HTTP server will store the binary data, somewhere (for<br>
instance<br>
locally), some how (for instance in a plain file), so that the<br>
data is<br>
later on accessible by a simple GET <a href="http://server1/myLinuxImage" target="_blank">http://server1/myLinuxImage</a><br>
<br>
In that case, this HTTP server could easily be a full fleshed<br>
Glance<br>
store.<br>
<br>
Questions :<br>
1) Has this been already discussed/proposed ? If so, could<br>
someone give<br>
me a pointer to this work ?<br>
2) Can I start working on this ? (the 2 main work items are :<br>
'add an<br>
add method to glance_store._drivers.http.__<u></u>Store' and 'add a<br>
delete<br>
method to glance_store._drivers.http.__<u></u>Store (HTTP DELETE<br>
method)'<br>
<br>
<br>
What is the difference between just calling the Glance API to upload<br>
an<br>
image, versus adding add() functionality to the HTTP image store?<br>
<br>
Best,<br>
-jay<br>
<br>
<br>
_____________________________<u></u>______________________________<u></u>_______________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="mailto:OpenStack-dev-request@lists.openstack.org" target="_blank">OpenStack-dev-request@lists.<u></u>openstack.org</a>?<br>
subject:unsubscribe<br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack-dev</a><br>
<br>
<br>
<br>
<br>
______________________________<u></u>______________________________<u></u>______________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="mailto:OpenStack-dev-request@lists.openstack.org" target="_blank">OpenStack-dev-request@lists.<u></u>openstack.org</a>?<br>
subject:unsubscribe<br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack-dev</a><br>
<br>
<br>
<br>
--<br>
@flaper87<br>
Flavio Percoco<br>
______________________________<u></u>______________________________<u></u>______________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.<u></u>openstack.org?subject:<u></u>unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack-dev</a><br>
<br>
<br>
<br>
</blockquote>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
______________________________<u></u>______________________________<u></u>______________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.<u></u>openstack.org?subject:<u></u>unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack-dev</a><br>
</blockquote>
<br>
<br>
-- <br>
@flaper87<br>
Flavio Percoco<br>
</div></div><br>__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div></div>