Open Stack

On 12/13/2018 7:27 AM, Chris Dent wrote:
> I guess in this case these tests were exposed by their failing, and
> it was only once investigating that you realized they weren't truly
> integration tests? Have you, Matt, got any ideas on how to find
> other non-integration tests that are being treated as integration
> which we could move to their own things? De-tangling the spaghetti
> is likely to reveal plenty of improvements but also plenty of areas
> that need more attention.

I haven't done a full audit anytime recent, no. I'm sure there are lots 
of tempest tests that are single-service, e.g. doing things in just 
cinder, glance, nova or neutron, which don't require any other services 
(nova might be the exception in some cases since we need at least glance 
and neutron for building a server). There was a time a few years ago 
where QA folk were working on pulling stuff like that out of tempest and 
moving it into the project trees if their functional testing would cover 
it, e.g. anything that just required the compute API and DB was a 
candidate to move into nova functional tests (like flavor and aggregates 
tests). However, interop tests are based on tempest so there are some 
tests you just can't remove because they are used by refstack.

-- 

Thanks,

Matt