Open Stack

Thanks for the confirmation, I agree use internal maintain CI whitelist is
a good way, and I confirmed with our CI folks that we already did so and
more can be removed per
Eric's test result below, so we will compare and remove unnecessary cases
to get more bandwidth to run 3rd party CI. thanks

Best Regards!

Kevin (Chen) Ji 纪 晨

Engineer, zVM Development, CSTL
Notes: Chen CH Ji/China/IBM at IBMCN   Internet: jichenjc at cn.ibm.com
Phone: +86-10-82451493
Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District,
Beijing 100193, PRC



From:	Ghanshyam Mann <gmann at ghanshyammann.com>
To:	"OpenStack Development Mailing List \\"
            <openstack-dev at lists.openstack.org>
Date:	09/07/2018 02:18 PM
Subject:	Re: [openstack-dev] [tempest][CI][nova compute] Skipping
            non-compute-driver tests






 ---- On Fri, 07 Sep 2018 04:41:32 +0900 Eric Fried <openstack at fried.cc>
wrote ----
 > Jichen-
 >
 >     That patch is not ever intended to merge; hope that was clear from
the
 > start :) It was just a proving ground to demonstrate which tests still
 > pass when there's effectively no compute driver in play.
 >
 >     We haven't taken any action on this from our end, though we have
done a
 > little brainstorming about how we would tool our CI to skip those tests
 > most (but not all) of the time. Happy to share our experiences with you
 > if/as we move forward with that.
 >
 >     Regarding the tempest-level automation, I certainly had z in mind
when
 > I was thinking about it. If you have the time and inclination to help
 > look into it, that would be most welcome.

Sorry for late response, somehow i missed this thread. As you mentioned and
noticed in your patch that there are ~700 tests which does not touch
compute driver. Most of them are from neutron-tempest-plugins or other
service tests. From Tempest compute tests, many of them are negative tests
or DB layer tests [1].

neutron-tempest-plugin or other service test you can always avoid to run
with regex. And i do not think compute negative or DB test will take much
time to run. But still if you want to avoid to run then, I think it is easy
to maintain a whitelist regex file on CI side which can run only compute
driver tests(61 in this case).

Tagging compute driver on tempest side is little hard to maintain i feel
and it can goes out of date very easily. If you have any other idea on
that, we can surly talk in PTG on this.

[1]
http://184.172.12.213/66/599066/5/check/nova-powervm-out-of-tree-pvm/a1b42d5/powervm_os_ci.html.gz


 >
 > Thanks,
 > efried
 >
 > On 09/06/2018 12:33 AM, Chen CH Ji wrote:
 > > I see the patch is still ongoing status and do you have a follow up
 > > plan/discussion for that? we are maintaining 2 CIs (z/VM and KVM on z)
 > > so skip non-compute related cases will be a good for 3rd part CI ..
thanks
 > >
 > > Best Regards!
 > >
 > > Kevin (Chen) Ji 纪 晨
 > >
 > > Engineer, zVM Development, CSTL
 > > Notes: Chen CH Ji/China/IBM at IBMCN Internet: jichenjc at cn.ibm.com
 > > Phone: +86-10-82451493
 > > Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian
 > > District, Beijing 100193, PRC
 > >
 > > Inactive hide details for Eric Fried ---09/04/2018 09:35:09
PM---Folks-
 > > The other day, I posted an experimental patch [1] withEric Fried
 > > ---09/04/2018 09:35:09 PM---Folks- The other day, I posted an
 > > experimental patch [1] with an effectively
 > >
 > > From: Eric Fried <openstack at fried.cc>
 > > To: "OpenStack Development Mailing List (not for usage questions)"
 > > <openstack-dev at lists.openstack.org>
 > > Date: 09/04/2018 09:35 PM
 > > Subject: [openstack-dev] [tempest][CI][nova compute] Skipping
 > > non-compute-driver tests
 > >
 > >
------------------------------------------------------------------------
 > >
 > >
 > >
 > > Folks-
 > >
 > > The other day, I posted an experimental patch [1] with an effectively
 > > empty ComputeDriver (just enough to make n-cpu actually start) to see
 > > how much of our CI would pass. The theory being that any tests that
 > > still pass are tests that don't touch our compute driver, and are
 > > therefore not useful to run in our CI environment. Because anything
that
 > > doesn't touch our code should already be well covered by generic
 > > dsvm-tempest CIs. The results [2] show that 707 tests still pass.
 > >
 > > So I'm wondering whether there might be a way to mark tests as being
 > > "compute driver-specific" such that we could switch off all the other
 > > ones [3] via a one-line conf setting. Because surely this has
potential
 > > to save a lot of CI resource not just for us but for other driver
 > > vendors, in tree and out.
 > >
 > > Thanks,
 > > efried
 > >
 > > [1]
https://review.openstack.org/#/c/599066/

 > > [2]
 > >
http://184.172.12.213/66/599066/5/check/nova-powervm-out-of-tree-pvm/a1b42d5/powervm_os_ci.html.gz

 > > [3] I get that there's still value in running all those tests. But it
 > > could be done like once every 10 or 50 or 100 runs instead of every
time.
 > >
 > >
__________________________________________________________________________
 > > OpenStack Development Mailing List (not for usage questions)
 > > Unsubscribe:
OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
 > >
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 > >
 > >
 > >
 > >
 > >
 > >
 > >
__________________________________________________________________________
 > > OpenStack Development Mailing List (not for usage questions)
 > > Unsubscribe:
OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
 > >
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 > >
 >
 >
__________________________________________________________________________
 > OpenStack Development Mailing List (not for usage questions)
 > Unsubscribe:
OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
 >
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 >



__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20180907/2b5d2565/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20180907/2b5d2565/attachment-0001.gif>