<html><body><p><font size="2">Thanks for the confirmation, I agree use internal maintain CI whitelist is a good way, and I confirmed with our CI folks that we already did so and more can be removed per</font><br><font size="2">Eric's test result below, so we will compare and remove unnecessary cases to get more bandwidth to run 3rd party CI. thanks</font><br><br><font size="2">Best Regards! <br><br>Kevin (Chen) Ji ¼Í ³¿<br><br>Engineer, zVM Development, CSTL<br>Notes: Chen CH Ji/China/IBM@IBMCN Internet: jichenjc@cn.ibm.com<br>Phone: +86-10-82451493<br>Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC </font><br><br><img width="16" height="16" src="cid:1__=8FBB0992DFA1EAB38f9e8a93df938690918c8FB@" border="0" alt="Inactive hide details for Ghanshyam Mann ---09/07/2018 02:18:45 PM--- ---- On Fri, 07 Sep 2018 04:41:32 +0900 Eric Fried <opens"><font size="2" color="#424282">Ghanshyam Mann ---09/07/2018 02:18:45 PM--- ---- On Fri, 07 Sep 2018 04:41:32 +0900 Eric Fried <openstack@fried.cc> wrote ---- > Jichen-</font><br><br><font size="2" color="#5F5F5F">From: </font><font size="2">Ghanshyam Mann <gmann@ghanshyammann.com></font><br><font size="2" color="#5F5F5F">To: </font><font size="2">"OpenStack Development Mailing List \\" <openstack-dev@lists.openstack.org></font><br><font size="2" color="#5F5F5F">Date: </font><font size="2">09/07/2018 02:18 PM</font><br><font size="2" color="#5F5F5F">Subject: </font><font size="2">Re: [openstack-dev] [tempest][CI][nova compute] Skipping non-compute-driver tests</font><br><hr width="100%" size="2" align="left" noshade style="color:#8091A5; "><br><br><br><tt><font size="2"><br><br><br> ---- On Fri, 07 Sep 2018 04:41:32 +0900 Eric Fried <openstack@fried.cc> wrote ---- <br> > Jichen-<br> > <br> > That patch is not ever intended to merge; hope that was clear from the<br> > start :) It was just a proving ground to demonstrate which tests still<br> > pass when there's effectively no compute driver in play.<br> > <br> > We haven't taken any action on this from our end, though we have done a<br> > little brainstorming about how we would tool our CI to skip those tests<br> > most (but not all) of the time. Happy to share our experiences with you<br> > if/as we move forward with that.<br> > <br> > Regarding the tempest-level automation, I certainly had z in mind when<br> > I was thinking about it. If you have the time and inclination to help<br> > look into it, that would be most welcome.<br><br>Sorry for late response, somehow i missed this thread. As you mentioned and noticed in your patch that there are ~700 tests which does not touch compute driver. Most of them are from neutron-tempest-plugins or other service tests. From Tempest compute tests, many of them are negative tests or DB layer tests [1].<br><br>neutron-tempest-plugin or other service test you can always avoid to run with regex. And i do not think compute negative or DB test will take much time to run. But still if you want to avoid to run then, I think it is easy to maintain a whitelist regex file on CI side which can run only compute driver tests(61 in this case). <br><br>Tagging compute driver on tempest side is little hard to maintain i feel and it can goes out of date very easily. If you have any other idea on that, we can surly talk in PTG on this. <br><br>[1] </font></tt><tt><font size="2"><a href="http://184.172.12.213/66/599066/5/check/nova-powervm-out-of-tree-pvm/a1b42d5/powervm_os_ci.html.gz">http://184.172.12.213/66/599066/5/check/nova-powervm-out-of-tree-pvm/a1b42d5/powervm_os_ci.html.gz</a></font></tt><tt><font size="2"><br><br> > <br> > Thanks,<br> > efried<br> > <br> > On 09/06/2018 12:33 AM, Chen CH Ji wrote:<br> > > I see the patch is still ongoing status and do you have a follow up<br> > > plan/discussion for that? we are maintaining 2 CIs (z/VM and KVM on z)<br> > > so skip non-compute related cases will be a good for 3rd part CI .. thanks<br> > > <br> > > Best Regards!<br> > > <br> > > Kevin (Chen) Ji ¼Í ³¿<br> > > <br> > > Engineer, zVM Development, CSTL<br> > > Notes: Chen CH Ji/China/IBM@IBMCN Internet: jichenjc@cn.ibm.com<br> > > Phone: +86-10-82451493<br> > > Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian<br> > > District, Beijing 100193, PRC<br> > > <br> > > Inactive hide details for Eric Fried ---09/04/2018 09:35:09 PM---Folks-<br> > > The other day, I posted an experimental patch [1] withEric Fried<br> > > ---09/04/2018 09:35:09 PM---Folks- The other day, I posted an<br> > > experimental patch [1] with an effectively<br> > > <br> > > From: Eric Fried <openstack@fried.cc><br> > > To: "OpenStack Development Mailing List (not for usage questions)"<br> > > <openstack-dev@lists.openstack.org><br> > > Date: 09/04/2018 09:35 PM<br> > > Subject: [openstack-dev] [tempest][CI][nova compute] Skipping<br> > > non-compute-driver tests<br> > > <br> > > ------------------------------------------------------------------------<br> > > <br> > > <br> > > <br> > > Folks-<br> > > <br> > > The other day, I posted an experimental patch [1] with an effectively<br> > > empty ComputeDriver (just enough to make n-cpu actually start) to see<br> > > how much of our CI would pass. The theory being that any tests that<br> > > still pass are tests that don't touch our compute driver, and are<br> > > therefore not useful to run in our CI environment. Because anything that<br> > > doesn't touch our code should already be well covered by generic<br> > > dsvm-tempest CIs. The results [2] show that 707 tests still pass.<br> > > <br> > > So I'm wondering whether there might be a way to mark tests as being<br> > > "compute driver-specific" such that we could switch off all the other<br> > > ones [3] via a one-line conf setting. Because surely this has potential<br> > > to save a lot of CI resource not just for us but for other driver<br> > > vendors, in tree and out.<br> > > <br> > > Thanks,<br> > > efried<br> > > <br> > > [1] </font></tt><tt><font size="2"><a href="https://review.openstack.org/#/c/599066/">https://review.openstack.org/#/c/599066/</a></font></tt><tt><font size="2"><br> > > [2]<br> > > </font></tt><tt><font size="2"><a href="http://184.172.12.213/66/599066/5/check/nova-powervm-out-of-tree-pvm/a1b42d5/powervm_os_ci.html.gz">http://184.172.12.213/66/599066/5/check/nova-powervm-out-of-tree-pvm/a1b42d5/powervm_os_ci.html.gz</a></font></tt><tt><font size="2"><br> > > [3] I get that there's still value in running all those tests. But it<br> > > could be done like once every 10 or 50 or 100 runs instead of every time.<br> > > <br> > > __________________________________________________________________________<br> > > OpenStack Development Mailing List (not for usage questions)<br> > > Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe<br> > > </font></tt><tt><font size="2"><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a></font></tt><tt><font size="2"><br> > > <br> > > <br> > > <br> > > <br> > > <br> > > <br> > > __________________________________________________________________________<br> > > OpenStack Development Mailing List (not for usage questions)<br> > > Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe<br> > > </font></tt><tt><font size="2"><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a></font></tt><tt><font size="2"><br> > > <br> > <br> > __________________________________________________________________________<br> > OpenStack Development Mailing List (not for usage questions)<br> > Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe<br> > </font></tt><tt><font size="2"><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a></font></tt><tt><font size="2"><br> > <br><br><br><br>__________________________________________________________________________<br>OpenStack Development Mailing List (not for usage questions)<br>Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe<br></font></tt><tt><font size="2"><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a></font></tt><tt><font size="2"><br><br></font></tt><br><br><BR>
</body></html>