Open Stack

I could do that, but I don't get the auto scaling that nodepool offers
since nodepool only inspects the zuul queue for needs and not the
Jenkins build queue. Unless there's a way to make nodepool monitor the
Jenkins build queue as well that I'm not aware of.

We're operating in a public cloud so we're resource constrained in
multiple ways.

* by budget
* by the number of nodes we can have operating at a given time
* by the bursty nature of our CI we can't just define keep N number of
VMs always available when that N needs to be shared between multiple
different slave definitions / queues.

-Andy-

On 02/24/2016 08:58 AM, Doug Wiegley wrote:
> I’m doing something similar, but am just specifying nodepool nodes
> on the child jobs, with artifact copying in Jenkins. I don’t think you need
> JClouds at that point. Jobs that use nodepool nodes don’t have to
> originate with zuul. Or am I missing your restriction?
> 
> doug
> 
> 
>> On Feb 24, 2016, at 10:12 AM, Andrew Grimberg <agrimberg at linuxfoundation.org> wrote:
>>
>> The problem with that is that we would then continue to have to rely on
>> JClouds to do slave management and we have two driving reasons for this
>> desire / need to switch to Zuul and Nodepool (or something similar)
>>
>> 1) Recent versions of the Gerrit Trigger plugin for Jenkins forget, or
>> do not update current triggers on JJB updates without someone either
>> manually clicking save in the job configuration UI or doing a full
>> restart of Jenkins. This is a bad thing for us, we're at nearly 2k jobs
>> managed via JJB many of which are generated by templates and a single
>> change can affect more than 100 jobs
>>
>> 2) JClouds is a single-threaded nightmare inside Jenkins and can easily
>> cause it to hang creation of new slaves, deletion of old slaves and
>> cause the Jenkins queue itself to not start jobs because a single job
>> got hung and JClouds gets confused. This along with it completely
>> forgetting about nodes that it started but they failed to connect for
>> some reason.
>>
>> So, while that sounds like a viable solution, it's not :(
>>
>> -Andy-
>>
>> On 02/24/2016 08:04 AM, Doug Wiegley wrote:
>>> Even if zuul doesn’t support the flow you want, you can also still
>>> use matrix, multijob, and downstream jobs on Jenkins itself to simulate the
>>> same behavior, with zuul just monitoring the parent job. Mix and match
>>> until you get the flow that you need.
>>>
>>> doug
>>>
>>>
>>>> On Feb 24, 2016, at 9:56 AM, Andrew Grimberg <agrimberg at linuxfoundation.org> wrote:
>>>>
>>>> Greetings,
>>>>
>>>> We're working on transitioning our Jenkins system off of Gerrit Trigger
>>>> and JClouds to Zuul and Nodepool and have run into a bit of a snag.
>>>>
>>>> We have some rather deep pipeline dependencies with our current
>>>> configurations where one job produces information and possibly artifacts
>>>> that are then passed on to one or more jobs downstream.
>>>>
>>>> Is this sort of configuration even possible with Zuul? I can't seem to
>>>> find anything in the documentation that would explain how to do this. If
>>>> it's not currently possible, what would need to be done to extend Zuul
>>>> to support something like this?
>>>>
>>>> As a simple example we're looking to something like this (please excuse
>>>> the bad ascii art)
>>>>
>>>>                               / Verify1 \
>>>> Gerrit event -- Zuul trigger --   Verify2  -- PipelineJob1 -- Response
>>>>                               \ Verify3 /
>>>>
>>>>
>>>> What the documentation is leading me to believe is that I can't do that
>>>> extra PipelineJob1 (or multiple in serial / parallel)
>>>>
>>>> -Andy-
>>>>
>>>> -- 
>>>> Andrew J Grimberg
>>>> Systems Administrator
>>>> The Linux Foundation
>>>>
>>>> _______________________________________________
>>>> OpenStack-Infra mailing list
>>>> OpenStack-Infra at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>>>
>>
>> -- 
>> Andrew J Grimberg
>> Systems Administrator
>> The Linux Foundation
>>
> 

-- 
Andrew J Grimberg
Systems Administrator
The Linux Foundation

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-infra/attachments/20160224/f869cfae/attachment.pgp>