[openstack-dev] [all][tc] Clarifying testing recommendation for interop programs
andrea.frittoli at gmail.com
Fri Jan 19 15:20:31 UTC 2018
On Fri, Jan 12, 2018 at 9:50 AM Luigi Toscano <ltoscano at redhat.com> wrote:
> On Thursday, 11 January 2018 23:52:00 CET Matt Riedemann wrote:
> > On 1/11/2018 10:36 AM, Colleen Murphy wrote:
> > > 1) All trademark-related tests should go in the tempest repo, in
> > > accordance
> > >
> > > with the original resolution. This would mean that even projects
> > > have
> > > never had tests in tempest would now have to add at least some of
> > > their
> > > black-box tests to tempest.
> > >
> > > The value of this option is that centralizes tests used for the Interop
> > > program in a location where interop-minded folks from the QA team can
> > > control them. The downside is that projects that so far have avoided
> > > having a dependency on tempest will now lose some control over the
> > > black-box tests that they use for functional and integration that would
> > > now also be used for trademark certification.
> > > There's also concern for the review bandwidth of the QA team - we can't
> > > expect the QA team to be continually responsible for an ever-growing
> > > of projects and their trademark tests.
> > How many tests are we talking about for designate and heat? Half a
> > dozen? A dozen? More?
> > If it's just a couple of tests per project it doesn't seem terrible to
> > have them live in Tempest so you get the "interop eye" on reviews, as
> > noted in your email. If it's a considerable amount, then option 2 seems
> > the best for the majority of parties.
> I would argue that it does not scale; what if some test is taken out from
> interoperability, and others are added? It would mean moving tests from one
> repository to another, with change of paths. I think that the solution 2,
> where the repository where a test belong and the functionality of a test
> not linked, is better.
This probably does not happen too often, but it does happen, and I agree
would make things easier to have interop and non-iterop tests in the same
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the OpenStack-dev