[OpenStack-Infra] Deep pipelining with zuul

Andrew Grimberg agrimberg at linuxfoundation.org
Wed Feb 24 16:12:17 UTC 2016


The problem with that is that we would then continue to have to rely on
JClouds to do slave management and we have two driving reasons for this
desire / need to switch to Zuul and Nodepool (or something similar)

1) Recent versions of the Gerrit Trigger plugin for Jenkins forget, or
do not update current triggers on JJB updates without someone either
manually clicking save in the job configuration UI or doing a full
restart of Jenkins. This is a bad thing for us, we're at nearly 2k jobs
managed via JJB many of which are generated by templates and a single
change can affect more than 100 jobs

2) JClouds is a single-threaded nightmare inside Jenkins and can easily
cause it to hang creation of new slaves, deletion of old slaves and
cause the Jenkins queue itself to not start jobs because a single job
got hung and JClouds gets confused. This along with it completely
forgetting about nodes that it started but they failed to connect for
some reason.

So, while that sounds like a viable solution, it's not :(

-Andy-

On 02/24/2016 08:04 AM, Doug Wiegley wrote:
> Even if zuul doesn’t support the flow you want, you can also still
> use matrix, multijob, and downstream jobs on Jenkins itself to simulate the
> same behavior, with zuul just monitoring the parent job. Mix and match
> until you get the flow that you need.
> 
> doug
> 
> 
>> On Feb 24, 2016, at 9:56 AM, Andrew Grimberg <agrimberg at linuxfoundation.org> wrote:
>>
>> Greetings,
>>
>> We're working on transitioning our Jenkins system off of Gerrit Trigger
>> and JClouds to Zuul and Nodepool and have run into a bit of a snag.
>>
>> We have some rather deep pipeline dependencies with our current
>> configurations where one job produces information and possibly artifacts
>> that are then passed on to one or more jobs downstream.
>>
>> Is this sort of configuration even possible with Zuul? I can't seem to
>> find anything in the documentation that would explain how to do this. If
>> it's not currently possible, what would need to be done to extend Zuul
>> to support something like this?
>>
>> As a simple example we're looking to something like this (please excuse
>> the bad ascii art)
>>
>>                                / Verify1 \
>> Gerrit event -- Zuul trigger --   Verify2  -- PipelineJob1 -- Response
>>                                \ Verify3 /
>>
>>
>> What the documentation is leading me to believe is that I can't do that
>> extra PipelineJob1 (or multiple in serial / parallel)
>>
>> -Andy-
>>
>> -- 
>> Andrew J Grimberg
>> Systems Administrator
>> The Linux Foundation
>>
>> _______________________________________________
>> OpenStack-Infra mailing list
>> OpenStack-Infra at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
> 

-- 
Andrew J Grimberg
Systems Administrator
The Linux Foundation

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-infra/attachments/20160224/17a8b962/attachment.pgp>


More information about the OpenStack-Infra mailing list