[openstack-dev] [qa][cinder][ceph] should Tempest tests the backend specific feature?
eharney at redhat.com
Tue May 2 13:40:48 UTC 2017
On 05/02/2017 01:42 AM, Ghanshyam Mann wrote:
> In Cinder, there are many features/APIs which are backend specific and
> will return 405 or 501 if same is not implemented on any backend .
> If such tests are implemented in Tempest, then it will break some gate
> where that backend job is voting. like ceph job in glance_store gate.
> There been many such cases recently where ceph jobs were broken due to
> such tests and recently it is for force-delete backup feature.
This problem was detected on the initial patch , where the ceph gate
failed with the 405 error, but the patch was merged anyway. Why are
there "many such cases" of these jobs getting broken?
> Reverting force-delete tests in . To resolve such cases at some
> extend, Jon is going to add a white/black list of tests which can run
> on ceph job  depends on what all feature ceph implemented. But this
> does not resolve it completely due to many reason like
> 1. External use of Tempest become difficult where user needs to know
> what all tests to skip for which backend
> 2. Tempest tests become too specific to backend.
> Now there are few options to resolve this:
> 1. Tempest should not tests such API/feature which are backend
> specific like mentioned by api-ref like.
Is the proposal here to test these features via the in-tree Cinder
tempest tests instead of from tempest itself, or just not test any
features in Cinder which have backend-specific differences?
> 2. Tempest test can be disabled/skip based on backend. - This is not
> good idea as it increase config options and overhead of setting those.
This option seems like the most straightforward way to get both good
test coverage and handle compatibility.
> 3. Tempest test can verify behavior with if else condition as per
> backend. This is bad idea and lose the test strength.
> IMO options 1 is better options. More feedback are welcome.
> ..1 https://developer.openstack.org/api-ref/block-storage/v3/?expanded=force-delete-a-backup-detail#force-delete-a-backup
> ..2 https://bugs.launchpad.net/glance/+bug/1687538
> ..3 https://review.openstack.org/#/c/461625/
> ..4 http://lists.openstack.org/pipermail/openstack-dev/2017-April/115229.html
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
More information about the OpenStack-dev