Open Stack

Good day Stackers.


I’d like to raise question about implementing custom pipelines for Zuul.
For those of you how’s pretty familiar with project-config and infra

itself it wouldn’t be a news that for now Zuul layout supports only few
pipelines types [1]
<https://github.com/openstack-infra/project-config/blob/17990d544f5162b9eebaa6b9781e7abbeab57b42/zuul/layout.yaml>
.

Most of OpenStack projects are maintaining more than one type of drivers
(for Nova - virt driver, Trove - datastore drivers,

Cinder - volume backends, etc.). And, as it can be seen, existing jenkins
check jobs are not wisely utilize infra resources.

This is a real problem, just remember end of every release - number of
check/recheck jobs is huge.


So, how can we utilize resources more wisely and run only needed check job?
Like we’ve been processing unstable new check jobs - putting them into

‘experimental’ pipeline. So why can’t we provide such ability for projects
to define their own pipelines?


For example, as code reviewer, i see that patch touches specific
functionality of Driver A and i know that project testing infrastructure
provides an ability

to examine specific workflow for Driver A. Then it seems to be more than
valid to post a comment on the review like “check driver-a”. As you can see

i want to ask gerrit to trig custom pipeline for given project. Let me
describe more concrete example from “real world”. In Trove we maintain 5
different

drivers for different datastores and it doesn’t look like a good thing to
run all check jobs against code that doesn’t actually touch any of existing
datastore

drivers (this is what we have right now [2]
<https://github.com/openstack-infra/project-config/blob/17990d544f5162b9eebaa6b9781e7abbeab57b42/zuul/layout.yaml#L1500-L1526>
).

           Now here comes my proposal. I’d like to extend existing Zuul
pipeline to support any of needed check jobs (see example of Triple-O, [3]
<https://github.com/openstack-infra/project-config/blob/master/zuul/layout.yaml#L174-L220>
).

           But, as i can, see there are possible problems with such
approach, so i also have an  alternative proposal to one above. The only
one way to deal with such

           approach is to use REGEX ‘files’ for job definitions (example:
requirements check job [4]
<https://github.com/openstack-infra/project-config/blob/master/zuul/layout.yaml#L659-L665>).
In this case we’d still maintain only one pipeline ‘experimental’

           for all second-priority jobs. To make small summary, two ways
were proposed:



   -

   Pipeline(s) per Project. Pros: reviewer can trig specific pipeline by
   himself. Cons: spamming status/zuul.
   -

   REGEX files per additional jobs.


Sorry, but i’m not able to describe all Pros/Cons for each of proposals.
So, if you know them, please help me to figure out them.


All thoughts/suggestions are welcome.


Kind regards

Denis M.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20141211/745978dd/attachment.html>