Open Stack

Scoring for 2018.01 is starting now.  Scoring is separate from testing.  The scoring says that the capability should be tested by the tests included in the guidelines.  But even if a capability is scored as being required, if there are no tests available to demonstrate the capability, that capability can't be included in required because of the lack of tests. If non-admin test(s) exist for a capability, it/the will be added to the list of guideline tests during the review period.

So, scoring can be done in parallel to writing the tests.  If the tests exist by the time scoring is finished, then life is easier.  From http://eavesdrop.openstack.org/meetings/interopwg/2017/interopwg.2017-09-20-16.01.log.html (16:48:24 on), first draft of the scores should be ready by 10/6.  That's when PTLs (who haven't already) and others should get involved and review/comment.  Final scores will be approved by the Summit.  
Once scoring is complete, the capabilities with passing scores, and their tests are reviewed and updated.

Operators and vendors start getting involved when the scores are official and first pass of the list of tests are out.  The downstream folks start running the tests and commenting in issues, config requirements, etc. 

In this particular instance, Refstack also needs to review its schema and design to ensure that it can also handle the microversion tests and reporting.  But, since Chris Hoge is the PTL, it's a pretty well known and visible issue.

So, does this help clarify anything?  Hope it helps.

--Rocky

> -----Original Message-----
> From: Matt Riedemann [mailto:mriedemos at gmail.com]
> Sent: Friday, September 29, 2017 1:08 PM
> To: interop-wg at lists.openstack.org
> Subject: [Interop-wg] Next steps for microversion testing
> 
> At the Queens PTG in Denver there was general agreement on starting to
> require some compute API microversions in the interop guidelines:
> 
> https://etherpad.openstack.org/p/InteropDenver2017PTG_Microversions
> 
> I hadn't seen anything in the ML yet so wanted to ask about status, as I don't
> attend the meetings.
> 
> There are four proposed microversions in that etherpad, but only one has an
> existing Tempest test.
> 
> Before working on writing Tempest tests for the others, what needs to
> happen (I don't know the process details).
> 
> Sounds like they need to be "scored", but who does that and/or when does
> it happen? Does a change need to be proposed somewhere and if so, should
> I do that?
> 
> I'd like to keep momentum on this given it was pretty positive at the PTG, so
> please let me know where I can help.
> 
> --
> 
> Thanks,
> 
> Matt
> 
> _______________________________________________
> Interop-wg mailing list
> Interop-wg at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/interop-wg