<div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Apr 23, 2013 at 4:46 PM, Doug Hellmann <span dir="ltr"><<a href="mailto:doug.hellmann@dreamhost.com" target="_blank">doug.hellmann@dreamhost.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote"><div><div class="h5">On Tue, Apr 23, 2013 at 1:09 PM, John Hopper <span dir="ltr"><<a href="mailto:john.hopper@jpserver.net" target="_blank">john.hopper@jpserver.net</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Hey Doug, we've been using falcon for a while now so I'll do my best to<br>
answer your questions.<br>
<div><br>
On 04/23/2013 10:47 AM, Doug Hellmann wrote:<br>
> On Tue, Apr 23, 2013 at 7:11 AM, Jay Pipes <<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>> wrote:<br>
><br>
>> This project (Falcon) is only about 4 months old. While it looks to be<br>
>> good solid code, it would be great if OpenStack projects could settle on<br>
>> a WSGI framework and everyone work towards the same end goal. By my<br>
>> count, to date OpenStack projects have used Bottle, Flask, Pecan, WSME,<br>
>> and custom-built frameworks.<br>
>><br>
><br>
> FTR, WSME is just being used for (de)serialization, so it could<br>
> theoretically be used with any of those other frameworks that handle the<br>
> request dispatching.<br>
><br>
><br>
>><br>
>> Why don't we get this into Oslo and at least make an attempt at driving<br>
>> the car in the same direction? If the performance benefits of Falcon are<br>
>> as advertised, I would think the OpenStack dev community would be eager<br>
>> to create a simple Oslo wrapper around the Falcon library and start to<br>
>> standardize on a single framework for WSGI handling.<br>
>><br>
><br>
> Those benchmarks look a little shallow. How does performance compare when<br>
> dispatching to the number of routes in the example app? Does searching a<br>
> long list of regexes scale in the same way as object-dispatch models?<br>
<br>
</div>Good question! I did a little bit of testing and it turns out that this<br>
is indeed a problem with the current routing dispatch mechanism. Adding<br>
100k routes did degrade performance considerably at 82 req/sec. This<br>
means that there's a scaling issue in the selection and dispatch<br>
algorithm, however, I don't see this being difficult to rectify. Also,<br>
being more reasonable with the number of routes added at 1K yielded<br>
acceptable performance results of 5700 req/sec. For reference, my GET<br>
request baseline pushed almost 17k req/sec.<br>
<div><br>
> What about development effort? How easy is it to add a new endpoint, or set<br>
> of endpoints? It looks like Falcon is using regex based dispatching,<br>
> similar to Flask. The problem we found with that in ceilometer is you can<br>
> have "dead spots" in your URL structure, where a URL is implied by a path<br>
> but doesn't actually have a handler registered. For example, the benchmark<br>
> app in the Falcon repo handles /hello/{account_id}/test, but does not say<br>
> what to do for /hello or /hello/{account_id}. That mistake would be more<br>
> obvious in an object-dispatch system.<br>
<br>
</div>These cases simply result in 404 returns - so yes, you can have dead<br>
spots but they're well handled. I would argue it's up to the API writer<br>
to make sure their resource structure is well written and correct.<br></blockquote><div><br></div></div></div><div>The other frameworks also return 404. The difference is it's more obvious that there may be a spot in the URL namespace that doesn't point to something while the handlers are being developed, so a reviewer can see it if the patch author doesn't.</div>
<div><div class="h5">
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div><br>
> The other aspect to consider is whether the framework includes the features<br>
> we need. Does it do content-type handling? Does it handle unicode in URLs<br>
> correctly? What about alternate message body encodings? How well does it<br>
> handle mal-formed requests? Can it handle streaming serialized data in a<br>
> response?<br>
<br>
</div>There are a lot of questions in there so I'll try to answer them in series.<br>
<br>
Content type handling is left to the API writer but it would be trivial<br>
to add a filter (see:<br>
<a href="https://github.com/racker/falcon/blob/master/tests/test_example.py#L37-L42" target="_blank">https://github.com/racker/falcon/blob/master/tests/test_example.py#L37-L42</a>)<br>
that's smart enough to introspect this header and provide the<br>
functionality desired.<br>
<br>
Unicode in the URL doesn't seem to route. I wrote a test containing しご<br>
と in the URI template and was unable to address it via CURL.<br>
<br>
Alternate message body encodings appear to be left up to the API writer<br>
as well but is not difficult to manage. This is probably best<br>
implemented in a manner similar to the way I described handing content<br>
types above.<br>
<br>
Malformed requests seem to be handled well in some cases but have me<br>
worried in other cases. It looks like falcon formats the response with<br>
what it extracted from the request sent in from the client. For example,<br>
sending in a request with an incorrect HTTP version resulted in the same<br>
incorrect version specified in the response. Obviously I would have to<br>
spend more time with the other obvious cases and maybe design a test<br>
suite for validation purposes.<br></blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><br>
Streaming is supported for responses. If you're serializing data you<br>
will have to manage that concern yourself by presenting the<br>
serialization as a stream object. See<br>
<a href="https://github.com/racker/falcon/blob/master/falcon/response.py#L24-L36" target="_blank">https://github.com/racker/falcon/blob/master/falcon/response.py#L24-L36</a><br>
<div><br></div></blockquote><div><br></div></div></div><div><div>So it sounds like Falcon is relatively fast at dispatching requests, but doesn't do nearly as much of the work for the developer as the other frameworks listed in the benchmarks.</div>
</div></div></div></div></blockquote><div><br></div><div style>Ryan Petrello, one of our devs, spent a couple of hours Tuesday working on Pecan performance. He was able to find at least 30% improvement using the Falcon benchmarks, just with low-hanging-fruit type changes. That code is now in the latest Pecan release (0.2.3).</div>
<div style><br></div><div style>Doug</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">
<div>
<div><br></div><div>Developer effort needs to be considered, too, IMHO.</div><div> <br></div></div><div class="im"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div>
> Correctly handling all of the WSGI spec while remaining interoperable with<br>
> other WSGI tools is fairly complex. WebOb has a *lot* of existing use and<br>
> testing. Are the request objects in Falcon as robust as WebOb?<br>
<br>
</div>If there's a test suite WebOb uses to validate how well it handled<br>
certain use cases valid or otherwise, I'd be happy to try and run<br>
similar tests against falcon. Granted, WebOb may have much of this<br>
already done in unit test, making comparing the two frameworks<br>
difficult. Put simply, given the maturity of WebOb in terms of projects<br>
using it, I would imagine that it is certainly more robust in that its<br>
had most of the edge cases vetted out by now.<br></blockquote><div><br></div></div><div>Unfortunately, I'm not aware of a compatibility test suite that can be run like that. OTOH, there's a huge community around WebOb and they have all implicitly agreed to contribute back solutions to issues that come up. I see no benefit to OpenStack for us to go off in a different direction and build something completely different. If WebOb performance needs to be improved, we should start by trying to contribute upstream.</div>
<span class="HOEnZb"><font color="#888888">
<div><br></div><div>Doug</div></font></span><div><div class="h5"><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div><div><br>
> The fact that we have several production-ready options for a framework,<br>
> Pecan or otherwise, make me wonder why we are spending time building<br>
> another one.<br>
><br>
> Doug<br>
><br>
><br>
>><br>
>> Best,<br>
>> -jay<br>
>><br>
>> On 04/23/2013 07:02 AM, Sandy Walsh wrote:<br>
>>> haha ... classic case of left hand not knowing what the right hand is<br>
>> doing.<br>
>>><br>
>>> Thanks!<br>
>>><br>
>>> -S<br>
>>> ________________________________________<br>
>>> From: Thierry Carrez [<a href="mailto:thierry@openstack.org" target="_blank">thierry@openstack.org</a>]<br>
>>> Sent: Tuesday, April 23, 2013 6:20 AM<br>
>>> To: <a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.openstack.org</a><br>
>>> Subject: Re: [openstack-dev] Pecan vs Falcon?<br>
>>><br>
>>> Sandy Walsh wrote:<br>
>>>> Anyone had any experience with this? Specs are impressive.<br>
>>>><br>
>>>> <a href="http://falconframework.org/" target="_blank">http://falconframework.org/</a><br>
>>><br>
>>> Given that it is developed by Kurt Griffiths @ Rackspace, and used in<br>
>>> Marconi, I'm pretty sure he has a strong opinion on it :)<br>
>>><br>
>>> --<br>
>>> Thierry Carrez (ttx)<br>
>>> Release Manager, OpenStack<br>
>>><br>
>>> _______________________________________________<br>
>>> OpenStack-dev mailing list<br>
>>> <a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
>>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
>>><br>
>>> _______________________________________________<br>
>>> OpenStack-dev mailing list<br>
>>> <a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
>>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
>>><br>
>><br>
>> _______________________________________________<br>
>> OpenStack-dev mailing list<br>
>> <a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
>><br>
><br>
><br>
><br>
> _______________________________________________<br>
> OpenStack-dev mailing list<br>
> <a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
><br>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</div></div></blockquote></div></div></div><br></div></div>
</blockquote></div><br></div></div>