[openstack-dev] [neutron] CI jobs take pretty long, can we improve that?

Doug Wiegley dougwig at parksidesoftware.com
Mon Mar 21 15:30:40 UTC 2016


> On Mar 21, 2016, at 5:40 AM, Ihar Hrachyshka <ihrachys at redhat.com> wrote:
> 
> Sean M. Collins <sean at coreitpro.com> wrote:
> 
>> Rossella Sblendido wrote:
>>> 2) multi-node jobs run for every patch set. Is that really what we want?
>>> They take pretty long. We could move them to a periodic job.
>> 
>> I would rather remove all the single-node jobs. Nova has been moving to
>> multinode jobs for their gate (if I recall correctly my
>> conversation with Dan Smith) and we should be moving in this direction
>> too. We should test Neutron the way it is deployed in production.
>> 
>> Also, who is really monitoring the periodic jobs? Truthfully? I know
>> there are some IPv6 jobs that are periodic and I'll be the first to
>> admit that I am not following them *at all*.
> 
> Well, stable maintainers track their periodic job failures. :) Email notifications when something starts failing help.
> 
>> 
>> So, my thinking is, unless it's running at the gate and inflicting pain
>> on people, it's not going to be a treated as a priority. Look at Linux
>> Bridge - serious race conditions that existed for years only
>> got fixed once I inflicted pain on all the Neutron devs by making it
>> voting and running on every patchset (sorry, not sorry).
> 
> I think there is still common ground between you and Rossella’s stances: the fact that we want to inflict gating pain does not mean that we want to execute every single job on each PS uploaded to gerrit. For some advanced and non-obvious checks [like partial grenade] the validation could be probably postponed till the patch hits the gate.
> 
> Yes, sometimes it will mean gate being reset due to the bad patch. This can be avoided in most of cases if reviewers and the author for a patch that potentially touches a specific scenario execute the jobs before hitting the gate with the patch [for example, if the job is in experimental set, it’s a matter of ‘check experimental’ before pressing W+1].

We have been pretty consciously moving neutron jobs to cause pain to *neutron* and not everyone else, which is the opposite of a “gate only” plan. Aside from that being against infra policy, I think I’m reading between the lines that folks are wanting faster iterations between patchsets. I note that the standard -full job is up to 55-65 minutes, from it’s old time of 40-45. Have we characterized why that’s so much slower now? Perhaps addressing that will bring down to the turn-around for all.

Thanks,
doug


> 
> Ihar
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




More information about the OpenStack-dev mailing list