Open Stack

Clint Byrum <clint at fewbar.com> writes:

> Excerpts from corvus's message of 2017-06-09 13:11:00 -0700:
>> Clark Boylan <cboylan at sapwetik.org> writes:
>> 
>> > I'm wary of this simply because it looks a lot like repeating
>> > OpenStack's (now failed) decision to stick web servers in a bunch of
>> > python processes then do cooperative multithreading with them along with
>> > all your application logic. It just gets complicated. I also think this
>> > underestimates the value of using tools people are familiar with (wsgi
>> > and flask) particularly if making it easy to jump in and building
>> > community is a goal.
>> 
>> I agree that mixing an asyncio based httpserver with application logic
>> using cooperative multithreading is not a good idea.  Happily that is
>> not the proposal.  The proposal is that the webserver be a separate
>> process from the rest of Zuul, it would be an independently scaleable
>> component, and *only* the webserver would use asyncio.
>> 
>
> I'm not totally convinced that having an HTTP service in the scheduler
> that gets proxied to when appropriate is the worst idea in the short term,
> since we already have one and it already works reasonably well with paste,
> we just want to get rid of paste faster than we can refactor it out by
> making a ZK backend.
>
> Even if we remove paste and create a web tier aiohttp thing, we end up
> writing most of what would be complex about doing it in-process in the
> scheduler. So, to tack gearman on top of that, versus just letting the
> reverse proxy do its job, seems like extra work.

What I'd like to get out of this conversation is a shared understanding
of what the web tier for Zuul should look like in the future, so that we
can know where we want to end up eventually, but *not* a set of
additional requirements for Zuul v3.0.  In other words, I think this is
a long-term, rather than short-term conversation.

The way I see it is that we're adding a bunch of new functionality to an
area of Zuul that we've traditionally kept very simple.  We're growing
from a simple JSON endpoint to support websockets, event injection via
hooks, and a full-blown API for historic data.

That last item in particular calls out for a real web framework.  Since
it is new work and has substantial interaction with the web framework,
it would be good to know what our end state is, so that folks working on
it can go ahead and head in that direction.

The other aspects, which are largely already implemented, can be ported
over in the fullness of time.

We do not need to change how we are doing webhooks or log streaming for
Zuul v3.0.

In fact, I imagine that at least initially, we would implement something
in openstack-infra like what you describe, Clint.  We will have an
Apache server which proxies status.json requests and webhooks to
zuul-scheduler, and proxies websocket requests to the streaming server.

As time permits, we can incorporate those into a comprehensive web
server with the framework we choose.

Does that sound like a good plan?

Does aiohttp alone fit the bill as Monty suggests, or do we need to
consider something else?

-Jim