[openstack-dev] [all][tc] Clarifying testing recommendation for interop programs

Ghanshyam Mann ghanshyammann at gmail.com
Fri Jan 19 04:10:00 UTC 2018

On Thu, Jan 18, 2018 at 11:22 PM, Graham Hayes <gr at ham.ie> wrote:
> On 18/01/18 16:25, Doug Hellmann wrote:
>> Excerpts from Graham Hayes's message of 2018-01-18 15:33:12 +0000:
> <snip/>
>> In the past the QA team agreed to accept trademark-related tests from
>> all projects in the tempest repo. Has that changed?
> There has not been an explict rejection but in all conversations the
> response has been "non core projects are outside the scope of tempest".
> Honestly, everytime we have tried to do something to core tempest
> we have had major pushback, and I want to clarify this before I or
> someone else put in the work of porting the base clients, getting CI
> configured*, and proposing the tests to tempest.

Yes, i do not remember that we have rejected the actual proposal or
patches or ever said that "QA team not going to accept interop needed
test inside Tempest even those are for other project than tempest
scope". Rather its been discussed that we need to check the situation
when new project going to be make in interop program. At least i
remember the discussion during Barcelona summit where there talked
about heat tests. We discussed we can check that based on when heat is
going to make in interop program.

Anyways let's analysts the current situation and work on best possible
solution than past things. I agree with Doug point about previous
resolution was passed and then why this new resolution ? and what is
not clear in previous resolution?. I think main issue is in
understanding the difference between  'trademark' program and 'adds-on
trademark' program.  Let me add the things point by point.

1. What is difference between "Trademark" program and "Adds-on
Trademark" program from interop certification? Can new projects go
under "Trademark" program.
    This will be helpful to understand the situation of keeping all
"Trademark" program tests and "Adds-on" program tests together or
separate. For example: any difference of doing their certification,
logo etc.

2.  As per previous resolution, and with all point of centralized test
location, expertise review, project independent ownership etc etc i
agree with option#1 and no "NO" to that now also. Now question comes
to practice implementation of that resolution which depends on 2

1. scale and number of program going to be in interop:
    As per current proposal, (i think its heat and designate and
around 20-30 tests as total) there is no issue for tempest team to
add/review/maintain them. But if that grows in number of program (than
number tests for e.x. having 50 tests of designate than 10 is not much
different things) and say 10 more program then it is difficult for QA
team to maintain those.

2. QA team review bandwidth.
    This is one of the big obstacle to extend the tempest scope. Like
other project, QA team face less contributors issues. Since 1-2 years,
I have been trying to attract new contributor in QA during upstream
training, mentorship program etc but people gets disappear after month
or so. Even all QA members are trying their best in this area but
unfortunately no success.

With both these factor i feel we can go with current resolution
(option#1- below solution) and help QA team also if situation gets
worst (QA team also human beings and need time to sleep :)).

1. QA team accept all interop defined program tests (tests only needed
by interop ).
2. Define a very clear process for collaboration between Interop, QA,
project team to help on adding/maintaining tests. Something like clear
guidelines of test req from interop,  MUST +1 from interop and project
3. If interop program grows more which become difficult to maintain by
QA team then accept the necessary change to resolution.


> - Graham
> * With zuulv3 this is *much* easier, so not as big a deal as it once was
> <snip/>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

More information about the OpenStack-dev mailing list