<div dir="ltr"><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;text-align:justify" id="docs-internal-guid-89d89364-3996-80a3-2a42-3e267a0ef3ee"><span style="font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">Good day Stackers.</span></p><br><br><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;text-indent:36pt;text-align:justify"><span style="font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">I’d like to raise question about implementing custom pipelines for Zuul. For those of you how’s pretty familiar with project-config and infra</span></p><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;text-indent:36pt;text-align:justify"><span style="font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline"> itself it wouldn’t be a news that for now Zuul layout supports only few pipelines types </span><a href="https://github.com/openstack-infra/project-config/blob/17990d544f5162b9eebaa6b9781e7abbeab57b42/zuul/layout.yaml" style="text-decoration:none"><span style="font-size:16px;font-family:"Times New Roman";color:rgb(17,85,204);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:underline;vertical-align:baseline">[1]</span></a><span style="font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">.</span></p><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;text-indent:36pt;text-align:justify"><span style="font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">Most of OpenStack projects are maintaining more than one type of drivers (for Nova - virt driver, Trove - datastore drivers,</span></p><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;text-indent:36pt;text-align:justify"><span style="font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline"> Cinder - volume backends, etc.). And, as it can be seen, existing jenkins check jobs are not wisely utilize infra resources.</span></p><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;text-indent:36pt;text-align:justify"><span style="font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">This is a real problem, just remember end of every release - number of check/recheck jobs is huge.</span></p><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;text-indent:36pt;text-align:justify"><span style="font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline"><br></span></p><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;text-indent:36pt;text-align:justify"><span style="font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">So, how can we utilize resources more wisely and run only needed check job? Like we’ve been processing unstable new check jobs - putting them into</span></p><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;text-indent:36pt;text-align:justify"><span style="font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">‘experimental’ pipeline. So why can’t we provide such ability for projects to define their own pipelines?</span></p><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;text-indent:36pt;text-align:justify"><span style="font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline"><br></span></p><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;text-indent:36pt;text-align:justify"><span style="font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">For example, as code reviewer, i see that patch touches specific functionality of Driver A and i know that project testing infrastructure provides an ability</span></p><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;text-indent:36pt;text-align:justify"><span style="font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">to examine specific workflow for Driver A. Then it seems to be more than valid to post a comment on the review like “check driver-a”. As you can see</span></p><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;text-indent:36pt;text-align:justify"><span style="font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">i want to ask gerrit to trig custom pipeline for given project. Let me describe more concrete example from “real world”. In Trove we maintain 5 different <br></span></p><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;text-indent:36pt;text-align:justify"><span style="font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">drivers for different datastores and it doesn’t look like a good thing to run all check jobs against code that doesn’t actually touch any of existing datastore</span></p><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;text-indent:36pt;text-align:justify"><span style="font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">drivers (this is what we have right now </span><a href="https://github.com/openstack-infra/project-config/blob/17990d544f5162b9eebaa6b9781e7abbeab57b42/zuul/layout.yaml#L1500-L1526" style="text-decoration:none"><span style="font-size:16px;font-family:"Times New Roman";color:rgb(17,85,204);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:underline;vertical-align:baseline">[2]</span></a><span style="font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">).</span></p><br><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;text-align:justify"><span style="font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">           Now here comes my proposal. I’d like to extend existing Zuul pipeline to support any of needed check jobs (see example of Triple-O, </span><a href="https://github.com/openstack-infra/project-config/blob/master/zuul/layout.yaml#L174-L220" style="text-decoration:none"><span style="font-size:16px;font-family:"Times New Roman";color:rgb(17,85,204);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:underline;vertical-align:baseline">[3]</span></a><span style="font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">).</span></p><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;text-align:justify"><span style="font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">           </span><span style="font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">But, as i can, see there are possible problems with such approach, so i also have an  alternative proposal to one above. The only one way to deal with such</span></p><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;text-align:justify"><span style="font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">           approach is to use REGEX ‘files’ for job definitions (example: requirements check job </span><a href="https://github.com/openstack-infra/project-config/blob/master/zuul/layout.yaml#L659-L665" style="text-decoration:none"><span style="font-size:16px;font-family:"Times New Roman";color:rgb(17,85,204);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:underline;vertical-align:baseline">[4]</span></a><span style="font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">). In this case we’d still maintain only one pipeline ‘experimental’</span></p><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;text-align:justify"><span style="font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">           for all second-priority jobs. </span><span style="font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">To make small summary, two ways were proposed:</span></p><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;text-align:justify"><span style="font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline"><br></span></p><ul style="margin-top:0pt;margin-bottom:0pt;margin-left:40px"><li style="list-style-type:disc;font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline"><p style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;text-align:justify"><span style="font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">Pipeline(s) per Project. Pros: reviewer can trig specific pipeline by himself. Cons: spamming status/zuul.</span></p></li><li style="list-style-type:disc;font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline"><p style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;text-align:justify"><span style="font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">REGEX files per additional jobs.</span></p></li></ul><br><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;text-indent:36pt;text-align:justify"><span style="font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">Sorry, but i’m not able to describe all Pros/Cons for each of proposals. So, if you know them, please help me to figure out them.<br></span></p><br><br><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;text-indent:36pt;text-align:justify"><span style="font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">All thoughts/suggestions are welcome.</span></p><br><br><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;text-align:justify"><span style="font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">Kind regards</span></p><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;text-align:justify"><span style="font-size:16px;font-family:"Times New Roman";color:rgb(0,0,0);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">Denis M.</span></p></div>