[openstack-dev] [tripleo] becoming third party CI (was: enabling third party CI)

Jeremy Stanley fungi at yuggoth.org
Thu Mar 10 23:24:44 UTC 2016


On 2016-03-10 16:09:44 -0500 (-0500), Dan Prince wrote:
> This seems to be the week people want to pile it on TripleO. Talking
> about upstream is great but I suppose I'd rather debate major changes
> after we branch Mitaka. :/
[...]

I didn't mean to pile on TripleO, nor did I intend to imply this was
something which should happen ASAP (or even necessarily at all), but
I do want to better understand what actual benefit is currently
derived from this implementation vs. a more typical third-party CI
(which lots of projects are doing when they find their testing needs
are not met by the constraints of our generic test infrastructure).

> With regards to Jenkins restarts I think it is understood that our job
> times are long. How often do you find infra needs to restart Jenkins?

We're restarting all 8 of our production Jenkins masters weekly at a
minimum, but generally more often when things are busy (2-3 times a
week). For many months we've been struggling with a thread leak for
which their development team has not seen as a priority to even
triage our bug report effectively. At this point I think we've
mostly given up on expecting it to be solved by anything other than
our upcoming migration off of Jenkins, but that's another topic
altogether.

> And regardless of that what if we just said we didn't mind the
> destructiveness of losing a few jobs now and then (until our job
> times are under the line... say 1.5 hours or so). To be clear I'd
> be fine with infra pulling the rug on running jobs if this is the
> root cause of the long running jobs in TripleO.

For manual Jenkins restarts this is probably doable (if additional
hassle), but I don't know whether that's something we can easily
shoehorn into our orchestrated/automated restarts.

> I think the "benefits are minimal" is bit of an overstatement. The
> initial vision for TripleO CI stands and I would still like to see
> individual projects entertain the option to use us in their gates.
[...]

This is what I'd like to delve deeper into. The current
implementation isn't providing you with any mechanism to prevent
changes which fail jobs running in the tripleo-test cloud from
merging to your repos, is it? You're still having to manually
inspect the job results posted by it? How is that particularly
different from relying on third-party CI integration?

As for other projects making use of the same jobs, right now the
only convenience I'm aware of is that they can add check-tripleo
pipeline jobs in our Zuul layout file instead of having you add it
to yours (which could itself reside in a Git repo under your
control, giving you even more flexibility over those choices). In
fact, with a third-party CI using its own separate Gerrit account,
you would be able to leave clear -1/+1 votes on check results which
is not possible with the present solution.

So anyway, I'm not saying that I definitely believe the third-party
CI route will be better for TripleO, but I'm not (yet) clear on what
tangible benefit you're receiving now that you lose by switching to
that model.
-- 
Jeremy Stanley



More information about the OpenStack-dev mailing list