Open Stack

On 19/11/17 03:08, Rabi Mishra wrote:
> Hi All,
> 
> As part of community goal[1] for Queens, we've completed the repo split 
> and created the new project[2].
> 
> The next objective to is to use the new plugin in our jobs. As we've 
> merged some changes after the split, I've synced them and fixed some 
> minor issues before using it in the gate jobs.
> 
> These are very small changes and I would suggest we review/land them on 
> priority (we don't have to keep syncing them again and again for broken 
> jobs). We should probably *not approve* any changes to integration tests 
> in heat before these go in.
> 
> - Changes in heat-tempest-plugin (sync  missing patches and fixes)
> https://review.openstack.org/#/q/project:openstack/heat-tempest-plugin+topic:sync_from_heat
> 
> - Use heat-tempest-plugin for integration jobs
> https://review.openstack.org/#/c/508112/
> 
> - Use heat-tempest-plugin for grenade job
> https://review.openstack.org/#/c/521246/
> 
> - Add heat integration jobs to heat-tempest-plugin check/gate queue
> https://review.openstack.org/#/c/521340/
>    I guess this would result in some issues, where we can't add any 
> changes to heat that breaks the existing tests, as the changes for both 
> projects would have a circular dependency (not sure how it works atm 
> with other plugins!).
> 
> - Remove plugin an integration tests from heat (This has -1 atm as 
> releasenote job is broken, waiting for infra to fix it)
> https://review.openstack.org/#/c/521263/ 
> <https://review.openstack.org/#/c/521263/>
> 
> I've also created an etherpad[3] to track these.
> 
> Also, the plugin project is expected to be branchless (we may not 
> backport these job changes to stable branches soon though), we've to 
> find  a  way to run additional tests for any new feature only on > 
> <release/branch>. AFAIK, other projects check api microversions 
> supported and without microversions in heat, may be we've find an 
> alternate way.

So from discussion on IRC today I learned that the plan agreed at the 
PTG was *not* to move all of heat_integrationtests to a separate repo, 
but just those tests that test the API and are likely to be used in the 
trademark program for verifying clouds.

In my view that effectively means just the Gabbi tests.

However, what is actually happening is that *all* of the 
integration/scenario tests are moving to the separate branchless repo. 
This looks to me like it will be a disaster for developer and reviewer 
productivity, and project quality[1]. (Other folks seem inclined to agree.)

The QA team assures us that the project-wide goal does not preclude 
keeping tests that are designed for Heat CI purposes (rather than cloud 
verification purposes) in-tree - even continuing to use tempest as a 
testrunner. However, it's not at all clear how to reconcile that with 
the goal description listing the evils of having an in-tree tempest 
plugin.[2] This appears to be the reason for the change in plan.

Does anybody remember what the mechanism for dealing with this that was 
agreed at the PTG was?

Basically we need a way to:

* Run a bunch of integration tests in Heat CI jobs
* against a full OpenStack cloud
* preferably using tempest as a testrunner, but this is optional
* without any danger of e.g. breaking tempest test discovery just 
because Heat was installed on the same machine as tempest

How do we do it?

thanks,
Zane.

[1] 
http://eavesdrop.openstack.org/irclogs/%23heat/%23heat.2017-11-29.log.html#t2017-11-29T15:17:00
[2] 
https://governance.openstack.org/tc/goals/queens/split-tempest-plugins.html#split-tempest-plugins-into-separate-repos-projects

> -- 
> Regards,
> Rabi Mishra
> 
> [1] 
> https://governance.openstack.org/tc/goals/queens/split-tempest-plugins.html
> [2] https://git.openstack.org/cgit/openstack/heat-tempest-plugin
> [3] https://etherpad.openstack.org/p/heat-tempest-plugin
> 
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>