Open Stack

On Tue, Dec 08, 2015 at 07:25:50PM +0000, Flavio Percoco wrote:
> Greetings,
> 
> I just reviewed a patch in tempest that proposed adding new tests for
> the Glance's task API. While I believe it's awesome that folks in the
> tempest team keep adding missing tests for projects, I think it'd be
> better for the tempest team, the project's team and defcore if we'd
> discuss these tests before they are worked on. This should help people
> avoid wasting time.

I agree a tasks API test isn't really something we want in tempest. I
actually didn't realize that there was a proposed test for this. (I've
been a bit lax in my tempest reviews lately)

> 
> I believe these cases are rare but the benefits of discussing missing
> tests across teams could also help prioritizing the work based on what
> the teams goals are, what the defcore team needs are, etc.
> 
> So, I'd like to start improving this process by inviting folks from
> the tempest team to join project's meeting whenever new tests are
> going to be worked on.

So, TBH I don't think this will really scale too well. The team of people
dedicated to working on tempest is relatively small compared the size of
OpenStack. I think taking the reverse approach and having project teams
taking a more hands on approach to tempest tests is the only way we'll be
able to scale here.

ML threads like this are actually a good approach on this front I think. It
raises awareness about a proposed test that has potential issues and discusses
it in an archived place.

> 
> I'd also like to invite PTLs (or anyone, really) from each team to go
> through what's in tempest and what's missing and help this team
> improve the test suite. Remember that these tests are also used by the
> defcore team and they are not important just for the CI but have an
> impact on other areas as well.
> 
> I'm doing the above for Glance and I can't stress enough how important
> it is for projects to do the same

A very big +1, on these 2 paragraphs. Tempest is used for much more than gating
and making sure we have what projects view as important is covered is important.
> 
> Do teams have a different workflow/process to increase tests coverage
> in tempest?
> 

I'm not aware of anything going on like this now. At one point in the past as
part of tempest development we used to have a single tempest BP per project per
cycle that project teams would use to collect and track test development for a
cycle. But, I haven't seen that in at least a year, likely longer.

-Matt Treinish
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20151208/969ef3a1/attachment.pgp>