On Tue, May 2, 2017 at 6:46 AM Ghanshyam Mann <ghanshyammann at gmail.com> wrote: > In Cinder, there are many features/APIs which are backend specific and > will return 405 or 501 if same is not implemented on any backend [1]. > If such tests are implemented in Tempest, then it will break some gate > where that backend job is voting. like ceph job in glance_store gate. > Having a test in Tempest is important for interoperability and API backward compatibility. As long as available features are discoverable and reported in a consistent way, it is possible for app developer to write application that will work fine against different backends. > > There been many such cases recently where ceph jobs were broken due to > such tests and recently it is for force-delete backup feature[2]. > Reverting force-delete tests in [3]. To resolve such cases at some > extend, Jon is going to add a white/black list of tests which can run > on ceph job [4] depends on what all feature ceph implemented. But this > does not resolve it completely due to many reason like > 1. External use of Tempest become difficult where user needs to know > what all tests to skip for which backend > 2. Tempest tests become too specific to backend. > > Now there are few options to resolve this: > 1. Tempest should not tests such API/feature which are backend > specific like mentioned by api-ref like[1]. > 2. Tempest test can be disabled/skip based on backend. - This is not > good idea as it increase config options and overhead of setting those. > Tempest has many options because we decide not to rely on discovery, i.e. we configure what we expect to find in the target cloud. I don't think we can use this as a factor to influence the decision in this case. > 3. Tempest test can verify behavior with if else condition as per > backend. This is bad idea and lose the test strength. > > IMO options 1 is better options. More feedback are welcome. > > ..1 > https://developer.openstack.org/api-ref/block-storage/v3/?expanded=force-delete-a-backup-detail#force-delete-a-backup > ..2 https://bugs.launchpad.net/glance/+bug/1687538 > ..3 https://review.openstack.org/#/c/461625/ > ..4 > http://lists.openstack.org/pipermail/openstack-dev/2017-April/115229.html > > -gmann > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20170503/ab1ffc6d/attachment.html>