Open Stack

On 05/02/2017 01:42 AM, Ghanshyam Mann wrote:
> In Cinder, there are many features/APIs which are backend specific and
> will return 405 or 501 if same is not implemented on any backend [1].
> If such tests are implemented in Tempest, then it will break some gate
> where that backend job is voting. like ceph job in glance_store gate.
> 
> There been many such cases recently where ceph jobs were broken due to
> such tests and recently it is for force-delete backup feature[2].

This problem was detected on the initial patch [5], where the ceph gate
failed with the 405 error, but the patch was merged anyway.  Why are
there "many such cases" of these jobs getting broken?

[5] https://review.openstack.org/#/c/332670/

> Reverting force-delete tests in [3]. To resolve such cases at some
> extend, Jon is going to add a white/black list of tests which can run
> on ceph job [4] depends on what all feature ceph implemented. But this
> does not resolve it completely due to many reason like
> 1. External use of Tempest become difficult where user needs to know
> what all tests to skip for which backend
> 2. Tempest tests become too specific to backend.
> 
> Now there are few options to resolve this:
> 1. Tempest should not tests such API/feature which are backend
> specific like mentioned by api-ref like[1].

Is the proposal here to test these features via the in-tree Cinder
tempest tests instead of from tempest itself, or just not test any
features in Cinder which have backend-specific differences?

> 2. Tempest test can be disabled/skip based on backend. - This is not
> good idea as it increase config options and overhead of setting those.

This option seems like the most straightforward way to get both good
test coverage and handle compatibility.

> 3. Tempest test can verify behavior with if else condition as per
> backend. This is bad idea and lose the test strength.
> 
> IMO options 1 is better options. More feedback are welcome.
> 
> ..1 https://developer.openstack.org/api-ref/block-storage/v3/?expanded=force-delete-a-backup-detail#force-delete-a-backup
> ..2 https://bugs.launchpad.net/glance/+bug/1687538
> ..3 https://review.openstack.org/#/c/461625/
> ..4 http://lists.openstack.org/pipermail/openstack-dev/2017-April/115229.html
> 
> -gmann
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>