Open Stack

On 08/12/15 15:31 -0500, Matthew Treinish wrote:
>On Tue, Dec 08, 2015 at 07:25:50PM +0000, Flavio Percoco wrote:
>> Greetings,
>>
>> I just reviewed a patch in tempest that proposed adding new tests for
>> the Glance's task API. While I believe it's awesome that folks in the
>> tempest team keep adding missing tests for projects, I think it'd be
>> better for the tempest team, the project's team and defcore if we'd
>> discuss these tests before they are worked on. This should help people
>> avoid wasting time.
>
>I agree a tasks API test isn't really something we want in tempest. I
>actually didn't realize that there was a proposed test for this. (I've
>been a bit lax in my tempest reviews lately)
>
>>
>> I believe these cases are rare but the benefits of discussing missing
>> tests across teams could also help prioritizing the work based on what
>> the teams goals are, what the defcore team needs are, etc.
>>
>> So, I'd like to start improving this process by inviting folks from
>> the tempest team to join project's meeting whenever new tests are
>> going to be worked on.
>
>So, TBH I don't think this will really scale too well. The team of people
>dedicated to working on tempest is relatively small compared the size of
>OpenStack. I think taking the reverse approach and having project teams
>taking a more hands on approach to tempest tests is the only way we'll be
>able to scale here.

Oh, I meant ocasionally. Whenever a missing test for an API is found,
it'd be easy enough for the implementer to sohw up at the meeting and
bring it up.

>
>ML threads like this are actually a good approach on this front I think. It
>raises awareness about a proposed test that has potential issues and discusses
>it in an archived place.

This works for me too! :)

Cheers,
Flavio

>
>>
>> I'd also like to invite PTLs (or anyone, really) from each team to go
>> through what's in tempest and what's missing and help this team
>> improve the test suite. Remember that these tests are also used by the
>> defcore team and they are not important just for the CI but have an
>> impact on other areas as well.
>>
>> I'm doing the above for Glance and I can't stress enough how important
>> it is for projects to do the same
>
>A very big +1, on these 2 paragraphs. Tempest is used for much more than gating
>and making sure we have what projects view as important is covered is important.
>>
>> Do teams have a different workflow/process to increase tests coverage
>> in tempest?
>>
>
>I'm not aware of anything going on like this now. At one point in the past as
>part of tempest development we used to have a single tempest BP per project per
>cycle that project teams would use to collect and track test development for a
>cycle. But, I haven't seen that in at least a year, likely longer.
>
>-Matt Treinish



-- 
@flaper87
Flavio Percoco