<div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, May 30, 2013 at 12:57 PM, Kurt Griffiths <span dir="ltr"><<a href="mailto:kurt.griffiths@rackspace.com" target="_blank">kurt.griffiths@rackspace.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="font-size:14px;font-family:Calibri,sans-serif;word-wrap:break-word">
<div>First, a few general thoughts on deprecating openstack.common.wsgi</div>
<ol>
<li>As Doug said, this isn't going to happen overnight; over time, as new versions of APIs come out, they should not use the OpenStack WSGI framework. I think that I'm also hearing that if a minor update to an API requires functionality not yet built into openstack.common.wsgi,
that should trigger a move to a new framework that <i>does</i> support that functionality, rather than continuing to add functionality to openstack.common.wsgi.</li></ol></div></blockquote><div style>I would be OK with that, but I do think it's a call we should make on a case-by-case basis. Fixing little bugs in place is still OK, as long as a given API is under support. </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="font-size:14px;font-family:Calibri,sans-serif;word-wrap:break-word"><ol><li>The fact that the Swift team just built something from scratch tells me there are still some concerns and/or gaps in functionality that have yet to be adequately addressed in webob. What's the plan to reconcile swob and webob?</li>
</ol><div class="im">
<div>> If you look very carefully at the benchmark scripts, you'll see that the numbers for falcon are artificially enhanced by the fact that the falcon test app is only looking at one regular expression when routing requests. Using more realistic number of
patterns in the routing list, in a random order, makes the difference in performance much smaller. </div>
<div><br>
</div>
</div><div>Actually, the number of routes tested by John were anything but realistic, if that's what you are referring to. With something more like 10-20 routes, which is what you would expect with a data API, Falcon still performs like a champ, and there's a tree-based
router in the works that would make even insanely large numbers of deep routes perform extremely well (although I'm not yet convinced there is a real-world use case for that).</div><div class="im">
<div><br>
</div>
<div>> There have also been changes in Pecan to improve its performance significantly (30% based on the benchmark app in the falcon repo).</div>
<div><br>
</div>
</div><div>+1. Rock on.</div>
<div><br>
</div>
<div>> I am much more interested in the ease of use. </div>
<div><br>
</div>
<div>For a management API, which most OpenStack APIs are, this makes total sense. On the other hand, for a data API, every microsecond counts in terms of latency a and throughput, particularly under highly concurrent workloads, so you tend to be biased more
towards performance vs. ease of use. That being said, I think it's a fallacy to say that you have to give up all ease-of-use to get great performance.</div></div></blockquote><div><br></div><div style>Fair point. And, for the record, I was comparing the ease of use of Pecan/WSME with the *current* framework, not Falcon. :-)</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="font-size:14px;font-family:Calibri,sans-serif;word-wrap:break-word">
<div><br>
</div>
<div>
<div>Ease of use is hard to quantify; some folks actually prefer Falcon's approach to building APIs, while others prefer Flask or Pecan or Bottle or X. That tells me that none of these experiences are a home run; perhaps it depends on what you are building.</div>
</div>
<div><br>
</div>
<div>Also, I think there are at least two different kinds of ease-of-use. One is development and the other is operational. The latter is one of the reasons the Swift team abandoned webob, if I'm not mistaken; WebOb had too much magic that made it hard to diagnose
errors in production. It would be great to see that addressed in a future WebOb version.</div></div></blockquote><div><br></div><div style>Indeed, it would be great to have some of those changes contributed upstream.</div>
<div style><br></div><div style>Doug</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="font-size:14px;font-family:Calibri,sans-serif;word-wrap:break-word">
<div><br>
</div>
<div>-Kurt (kgriffs)</div>
<div><br>
</div>
<div><br>
</div>
<span>
<div style="border-right:medium none;padding-right:0in;padding-left:0in;padding-top:3pt;text-align:left;font-size:11pt;border-bottom:medium none;font-family:Calibri;border-top:#b5c4df 1pt solid;padding-bottom:0in;border-left:medium none">
<span style="font-weight:bold">From: </span>Doug Hellmann <<a href="mailto:doug.hellmann@dreamhost.com" target="_blank">doug.hellmann@dreamhost.com</a>><br>
<span style="font-weight:bold">Reply-To: </span>OpenStack Dev <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.openstack.org</a>><br>
<span style="font-weight:bold">Date: </span>Thursday, May 30, 2013 9:55 AM<br>
<span style="font-weight:bold">To: </span>OpenStack Dev <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.openstack.org</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [openstack-dev] [oslo] deprecating openstack.common.wsgi<br>
</div><div><div class="h5">
<div><br>
</div>
<div>
<div>
<div dir="ltr"><br>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Thu, May 30, 2013 at 9:16 AM, Jarret Raim <span dir="ltr">
<<a href="mailto:jarret.raim@rackspace.com" target="_blank">jarret.raim@rackspace.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div style="font-size:14px;font-family:Calibri,sans-serif;word-wrap:break-word">
<div>
<div>
<div>
<div>Both Logging (Meniscus) and Key Management (Barbican) are using Falcon (<a href="http://falconframework.org/" target="_blank">http://falconframework.org/</a>) right now. I'm not adverse to a change if needed, but have we addressed the performance issues
from the Falcon page? If Pecan serves 9ish times fewer requests, that seems like a rather large red flag.</div>
</div>
</div>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>If you look very carefully at the benchmark scripts, you'll see that the numbers for falcon are artificially enhanced by the fact that the falcon test app is only looking at one regular expression when routing requests. Using more realistic number
of patterns in the routing list, in a random order, makes the difference in performance much smaller. This was discussed somewhat in the previous thread on the topic on this mailing list (<a href="http://lists.openstack.org/pipermail/openstack-dev/2013-April/007709.html" target="_blank">http://lists.openstack.org/pipermail/openstack-dev/2013-April/007709.html</a>).
There have also been changes in Pecan to improve its performance significantly (30% based on the benchmark app in the falcon repo).</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div style="font-size:14px;font-family:Calibri,sans-serif;word-wrap:break-word">
<div>
<div>
<div></div>
</div>
</div>
<div><br>
</div>
<div>One benchmark is not enough data to make any decisions, but I didn't see any discussion of the performance of Pecan / WSME in the etherpad from the design summit. Has anyone done any performance testing of the proposed set up? </div>
</div>
</blockquote>
<div><br>
</div>
<div>I am much more interested in the ease of use, since I don't see receiving and responding to the web request as the most expensive part of most API calls (especially not for anything that hits the database or makes RPC calls). We obviously don't
want to ignore performance, but I would rather put effort into improving an existing project that is very easy to use than into building something new from scratch.</div>
<div><br>
</div>
<div>Doug</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div style="font-size:14px;font-family:Calibri,sans-serif;word-wrap:break-word">
<div><br>
</div>
<div><br>
</div>
<div>Jarret </div>
<div><br>
</div>
<div><br>
</div>
<span>
<div style="border-width:1pt medium medium;border-style:solid none none;padding:3pt 0in 0in;text-align:left;font-size:11pt;font-family:Calibri;border-top-color:rgb(181,196,223)">
<span style="font-weight:bold">From: </span>Doug Hellmann <<a href="mailto:doug.hellmann@dreamhost.com" target="_blank">doug.hellmann@dreamhost.com</a>><br>
<span style="font-weight:bold">Reply-To: </span>OpenStack List <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.openstack.org</a>><br>
<span style="font-weight:bold">Date: </span>Wednesday, May 29, 2013 7:07 PM<br>
<span style="font-weight:bold">To: </span>OpenStack List <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.openstack.org</a>><br>
<span style="font-weight:bold">Subject: </span>[openstack-dev] [oslo] deprecating openstack.common.wsgi<br>
</div>
<div>
<div>
<div><br>
</div>
<div>
<div>
<div dir="ltr">I have -2ed several changes to the wsgi module in oslo-incubator in the last few weeks, and I wanted to bring it up on the list to explain the reasons more broadly.
<div><br>
</div>
<div>At the last summit we agreed that the goal was to start deprecating our home-grown WSGI service implementation in favor of moving to a new set of tools called Pecan and WSME (<a href="https://etherpad.openstack.org/havana-common-wsgi" target="_blank">https://etherpad.openstack.org/havana-common-wsgi</a>).
While I understand that we can't just drop everything and build new API servers, I do feel strongly that we should not continue developing new features for the module we are deprecating.</div>
<div><br>
</div>
<div>I don't see any other projects with "wsgi" in their openstack-common.conf, nor did I find any copies of openstack/common/wsgi.py in any other projects. I did find several wsgi.py files in different directories in the projects. I take that to mean that
we are/were still in the process of unifying the various forked versions of the module, and I propose that we just not bother doing more work on that. Since no project is already using Oslo's wsgi module, I'm not sure it makes sense to bring bug fixes into
Oslo, either. If I've missed something, and projects are in fact using the module, then we should continue to deal with bug fixes.<br>
<div><br>
</div>
<div>To officially indicate the module as deprecated, Mark suggested moving it to openstack/common/deprecated/wsgi.py. I have done that in changeset <a href="https://review.openstack.org/#/c/30972/" target="_blank">https://review.openstack.org/#/c/30972/</a>.
As the commit log message mentions, there is still some work to be done to remove the dependency between one or two of the middleware classes and the now deprecated module. I would appreciate any ideas about the best way to do that from folks who know about
those modules.<br>
<div><br>
</div>
<div>As we move to the new tools, I am sure we will find new bits of code related to setting up those projects that we can share. For example, the code to configure the keystone middleware for an API service shouldn't need to be repeated across all of the projects.
We should start a new module to hold those pieces, taken from the projects that are starting to use Pecan and WSME already. I volunteer to be the primary maintainer for that new module within Oslo.</div>
<div><br>
</div>
<div>Doug</div>
<div><br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</span></div>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</div></div></span>
</div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div></div>