[OpenStack-Infra] jenkins is stalled

Jeremy Stanley fungi at yuggoth.org
Sun Jul 28 00:46:09 UTC 2013


On 2013-07-26 16:34:07 +0000 (+0000), Sean Roberts wrote:
> lots of jobs queued, graph shows no jenkins nodes available

I believe this got discussed on IRC, but I'm following up for the
benefit of other subscribers because I don't see any replies
on-list.

Jenkins wasn't stalled, but it was working its way through a fairly
large volume of jobs. In general if you look at the "gate" pipeline
representation at http://status.openstack.org/zuul/ you shouldn't
see the same change at the top of that column for more than 60-90
minutes. If you see a job which you think may be stalled, you can
click on it and then follow the link to its real-time console output
in the Jenkins WebUI (keep in mind that some jobs like testr-based
unit test runs may not print anything until the unit tests complete,
so just because that console log's not updated for 15-30 minutes
also doesn't necessarily mean anything's wrong).

If you're not seeing votes for a while on changes which haven't been
approved yet, it may be because the gate is busy. Prioritization of
the gate got added recently; now when there aren't enough resources
to go around, jobs in the gate pipeline get first dibs on them. This
condition can be seen when all jobs in the check pipeline are
completed or queued while the only jobs in progress for a given
slave class (devstack, python2.6, python3.3, other) seem to be in
the gate pipeline.

I agree the real-time experience of the status interface is still
pretty close to watching paint dry (very nice looking paint!), but
there are upcoming changes to push far more progress detail into the
interface very soon.
-- 
Jeremy Stanley



More information about the OpenStack-Infra mailing list