Open Stack

On Thu, May 30, 2013 at 12:57 PM, Kurt Griffiths <
kurt.griffiths at rackspace.com> wrote:

>  First, a few general thoughts on deprecating openstack.common.wsgi
>
>    1. As Doug said, this isn't going to happen overnight; over time, as
>    new versions of APIs come out, they should not use the OpenStack WSGI
>    framework. I think that I'm also hearing that if a minor update to an API
>    requires functionality not yet built into openstack.common.wsgi, that
>    should trigger a move to a new framework that *does* support that
>    functionality, rather than continuing to add functionality
>    to openstack.common.wsgi.
>
> I would be OK with that, but I do think it's a call we should make on a
case-by-case basis. Fixing little bugs in place is still OK, as long as a
given API is under support.

>
>    1. The fact that the Swift team just built something from scratch
>    tells me there are still some concerns and/or gaps in functionality that
>    have yet to be adequately addressed in webob. What's the plan to reconcile
>    swob and webob?
>
> > If you look very carefully at the benchmark scripts, you'll see that the
> numbers for falcon are artificially enhanced by the fact that the falcon
> test app is only looking at one regular expression when routing requests.
> Using more realistic number of patterns in the routing list, in a random
> order, makes the difference in performance much smaller.
>
>  Actually, the number of routes tested by John were anything but
> realistic, if that's what you are referring to. With something more like
> 10-20 routes, which is what you would expect with a data API, Falcon still
> performs like a champ, and there's a tree-based router in the works that
> would make even insanely large numbers of deep routes perform extremely
> well (although I'm not yet convinced there is a real-world use case for
> that).
>
>  > There have also been changes in Pecan to improve its performance
> significantly (30% based on the benchmark app in the falcon repo).
>
>  +1. Rock on.
>
>  > I am much more interested in the ease of use.
>
>  For a management API, which most OpenStack APIs are, this makes total
> sense. On the other hand, for a data API, every microsecond counts in terms
> of latency a and throughput, particularly under highly concurrent
> workloads, so you tend to be biased more towards performance vs. ease of
> use. That being said, I think it's a fallacy to say that you have to give
> up all ease-of-use to get great performance.
>

Fair point. And, for the record, I was comparing the ease of use of
Pecan/WSME with the *current* framework, not Falcon. :-)


>
>  Ease of use is hard to quantify; some folks actually prefer Falcon's
> approach to building APIs, while others prefer Flask or Pecan or Bottle or
> X. That tells me that none of these experiences are a home run; perhaps it
> depends on what you are building.
>
>  Also, I think there are at least two different kinds of ease-of-use. One
> is development and the other is operational. The latter is one of the
> reasons the Swift team abandoned webob, if I'm not mistaken; WebOb had too
> much magic that made it hard to diagnose errors in production. It would be
> great to see that addressed in a future WebOb version.
>

Indeed, it would be great to have some of those changes contributed
upstream.

Doug


>
>  -Kurt (kgriffs)
>
>
>   From: Doug Hellmann <doug.hellmann at dreamhost.com>
> Reply-To: OpenStack Dev <openstack-dev at lists.openstack.org>
> Date: Thursday, May 30, 2013 9:55 AM
> To: OpenStack Dev <openstack-dev at lists.openstack.org>
> Subject: Re: [openstack-dev] [oslo] deprecating openstack.common.wsgi
>
>
>
>
> On Thu, May 30, 2013 at 9:16 AM, Jarret Raim <jarret.raim at rackspace.com>wrote:
>
>>   Both Logging (Meniscus) and Key Management (Barbican) are using Falcon
>> (http://falconframework.org/) right now. I'm not adverse to a change if
>> needed, but have we addressed the performance issues from the Falcon page?
>> If Pecan serves 9ish times fewer requests, that seems like a rather large
>> red flag.
>>
>
>  If you look very carefully at the benchmark scripts, you'll see that the
> numbers for falcon are artificially enhanced by the fact that the falcon
> test app is only looking at one regular expression when routing requests.
> Using more realistic number of patterns in the routing list, in a random
> order, makes the difference in performance much smaller. This was discussed
> somewhat in the previous thread on the topic on this mailing list (
> http://lists.openstack.org/pipermail/openstack-dev/2013-April/007709.html).
> There have also been changes in Pecan to improve its performance
> significantly (30% based on the benchmark app in the falcon repo).
>
>
>>
>>  One benchmark is not enough data to make any decisions, but I didn't
>> see any discussion of the performance of Pecan / WSME in the etherpad from
>> the design summit. Has anyone done any performance testing of the proposed
>> set up?
>>
>
>  I am much more interested in the ease of use, since I don't see
> receiving and responding to the web request as the most expensive part of
> most API calls (especially not for anything that hits the database or makes
> RPC calls). We obviously don't want to ignore performance, but I would
> rather put effort into improving an existing project that is very easy to
> use than into building something new from scratch.
>
>  Doug
>
>
>>
>>
>>  Jarret
>>
>>
>>   From: Doug Hellmann <doug.hellmann at dreamhost.com>
>> Reply-To: OpenStack List <openstack-dev at lists.openstack.org>
>> Date: Wednesday, May 29, 2013 7:07 PM
>> To: OpenStack List <openstack-dev at lists.openstack.org>
>> Subject: [openstack-dev] [oslo] deprecating openstack.common.wsgi
>>
>>   I have -2ed several changes to the wsgi module in oslo-incubator in
>> the last few weeks, and I wanted to bring it up on the list to explain the
>> reasons more broadly.
>>
>>  At the last summit we agreed that the goal was to start deprecating our
>> home-grown WSGI service implementation in favor of moving to a new set of
>> tools called Pecan and WSME (
>> https://etherpad.openstack.org/havana-common-wsgi). While I understand
>> that we can't just drop everything and build new API servers, I do feel
>> strongly that we should not continue developing new features for the module
>> we are deprecating.
>>
>>  I don't see any other projects with "wsgi" in their
>> openstack-common.conf, nor did I find any copies of
>> openstack/common/wsgi.py in any other projects. I did find several wsgi.py
>> files in different directories in the projects. I take that to mean that we
>> are/were still in the process of unifying the various forked versions of
>> the module, and I propose that we just not bother doing more work on that.
>> Since no project is already using Oslo's wsgi module, I'm not sure it makes
>> sense to bring bug fixes into Oslo, either. If I've missed something, and
>> projects are in fact using the module, then we should continue to deal with
>> bug fixes.
>>
>>  To officially indicate the module as deprecated, Mark suggested moving
>> it to openstack/common/deprecated/wsgi.py. I have done that in changeset
>> https://review.openstack.org/#/c/30972/. As the commit log message
>> mentions, there is still some work to be done to remove the dependency
>> between one or two of the middleware classes and the now deprecated module.
>> I would appreciate any ideas about the best way to do that from folks who
>> know about those modules.
>>
>>  As we move to the new tools, I am sure we will find new bits of code
>> related to setting up those projects that we can share. For example, the
>> code to configure the keystone middleware for an API service shouldn't need
>> to be repeated across all of the projects. We should start a new module to
>> hold those pieces, taken from the projects that are starting to use Pecan
>> and WSME already. I volunteer to be the primary maintainer for that new
>> module within Oslo.
>>
>>  Doug
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130530/7de2d28f/attachment-0001.html>