[openstack-dev] [qa][cinder][ceph] should Tempest tests the backend specific feature?

Sean McGinnis sean.mcginnis at gmx.com
Wed May 3 13:31:06 UTC 2017

On Wed, May 03, 2017 at 01:14:18PM +0000, Andrea Frittoli wrote:
> > >
> > Now that this is all in place, I think it's working well and I would like
> > to see it continue that way. IMO, tempest proper should not have anything
> > that isn't universally applicable to real world deployments. Not just for
> > things like Ceph, but things like the manage/unmanage backend specific
> > tests that were added and broke a large majority of third party CI.
> >
> Is there a policy in Cinder that a backend must implement a certain set of
> APIs? If so we could think of testing only that set of APIs in Tempest, so
> that
> any app developer knows that he/she can rely on that minimum set of APIs.

Yes, there is a minimum feature set that all backend drivers must support.
That base functionality can be found here [0] and here [1].

[0] https://docs.openstack.org/developer/cinder/devref/drivers.html#core-functionality
[1] https://github.com/openstack/cinder/blob/master/cinder/interface/volume_driver.py

The issue we've had with some of the tempest tests isn't actually around
whether the given backend supports the API. It's the way the API is used
that becomes a challenge.

I'll use the manage/unmanage snapshot one as an example. This is not one
of the required APIs, but many backends do support it. But the way this
API works is you are basically saying "manage this snapshot that you
identify as X", where "X" can be different things depending on the volume
driver being used. It's the storage device's native way of identifying
a snapshot, which may or may not be our Cinder snapshot ID.

So that test was added based on the way the LVM driver works for this.
That part is great, we get some more code coverage in the gate. But then
each volume driver that uses a different ID had to first a) start failing
CI, b) troubleshoot what happened to cause this new failure, c) reconfig
their CI to skip this test. Repeat cycle for each test added that does
something specific to how one or a small subset of backends work.

More information about the OpenStack-dev mailing list