Open Stack

On Mon, Sep 21, 2015 at 02:01:06PM +1200, Steve Baker wrote:
> On 20/09/15 20:24, Qiming Teng wrote:
> >Speaking of adding tests, we need hands on improving Heat API tests in
> >Tempest [1]. The current test cases there is a weird combination of API
> >tests, resource type tests, template tests etc. If we decide to move
> >functional tests back to individual projects, some test cases may need
> >to be deleted from tempest.
> >
> >Another important reason of adding API tests into Tempest is because
> >the orchestration service is assessed [2] by the DefCore team using
> >tests in Tempest, not in-tree test cases.
> >
> >The heat team has done a lot (and killed a lot) work to make the API as
> >stable as possible. Most of the time, there would be nothing new for
> >testing. The API surface tests may become nothing but waste of time if
> >we keep running them for every single patch.
> Thanks for raising this. Wherever they live we do need a dedicated
> set of tests which ensure the REST API is fully exercised.
> >So... my suggestions:
> >
> >- Remove unnecessary tests in Tempest;
> agreed
> >- Stop adding API tests to Heat locally;
> >- Add API tests to Tempest instead, in an organized way. (refer to [3])
> I would prefer an alternative approach which would result in the
> same end state:
> - port heat_integrationtests to tempest-lib
> - build a suite of REST API tests in heat_integrationtests
> - work with defcore to identify which heat_integrationtests tests to
> move to tempest

Sounds a reasonable approach. Thanks.

Regards,
 Qiming

> >[1]
> >http://git.openstack.org/cgit/openstack/tempest/tree/tempest/api/orchestration/
> >[2] https://review.openstack.org/#/c/216983/
> >[3] https://review.openstack.org/#/c/210080/
> >