Open Stack

On 2014-01-09 14:59:45 +0800 (+0800), James E. Blair wrote:
[...]
> I'd like to move the Zuul git merging component into a separate process
> that can be located on a separate host (or hosts) and scaled out.
> 
> The current zuul-server would continue to manage the queue and launch
> jobs, but as it processes the queue and decides which changes should be
> composed and built into zuul git refs, it would package the info about
> each ref and put it on the gearman queue as a work item.  An instance of
> the new component (zuul-merger) would fetch that job and fetch the
> needed refs from Gerrit, and merge them.  It would also serve the
> resulting git repo in the same way that Zuul does now.
> 
> Zuul would not have to wait for a response before continuing to process
> the queue, and since it's not doing any actual work, will be able to
> move through the queue _much_ faster than currently.  Once Zuul _does_
> receive a completion response from a zuul-merger, it can then launch the
> jobs for that change.  It will pass the URL for that particular
> zuul-merger (as ZUUL_URL) to the jobs so that they know from which
> merger to fetch the zuul ref.  We can also use the cancel job
> functionality in gearman if Zuul decides to reorder the queue.
[...]

Aside from a little extra latency queuing each change, I see no
obvious downsides. I expect it should overcome the current scaling
problem nicely.
-- 
Jeremy Stanley