[openstack-dev] [OpenStack][Cinder] Driver qualification

Mark Washenberger mark.washenberger at markwash.net
Tue Jul 30 20:36:28 UTC 2013

In general, I think this is fantastic and I know that Glance would benefit
from something similar.

I have a few opinions I want to share:

- I'd love to use this as a way to OpenStack-validate plugins that live
outside of the project--and then kick most plugins out of the project to
some other semi-canonical area.
- I don't think the test infrastructure should depend wholly on
devstack--while that makes a ton of sense for plugins that depend on other
OpenStack services working, I think its pretty crufty that we have plugins
that depend on other openstack services. I'd rather not to weigh the good
down with the bad.

Thanks for creating this discussion!

On Tue, Jul 30, 2013 at 12:04 PM, John Griffith <john.griffith at solidfire.com
> wrote:

> On Tue, Jul 30, 2013 at 11:47 AM, Walter A. Boring IV <
> walter.boring at hp.com> wrote:
>>  So how frequently would this be required and when do the results need
>> to be provided?
> To start I'd like to propose any new driver, and then each driver upon
> milestone release, or at the very least each cycle.
>>     I generally think this is a good idea, but as well all know that the
>> project seems to be in flux at times around the milestones.  Especially
>> around G3 when we changed the way the CONF object was accessed, almost all
>> drivers failed to work right after G3 when cinder.conf was configured with
>> a multi-backend setup.  It took a week or so after G3 before everyone
> IMO this is exactly why something like this would actually be more
> beneficial.
>>  was able to go through drivers and clean them up and get them to work in
>> this scenario.   I'm not trying to be a wet blanket on this idea, but I
>> think we just have to be careful.   I have no problem running my drivers
>> through this and providing the results.  What happens when something
>> changes in cinder that causes drivers to fail and the test runs fail?   How
>> long
> We should be extra careful to keep this sort of thing from happening, but
> again without doing any sort of public testing/results here we run the risk
> of nobody knowing it's broken until a customer encounters it.  I'm also not
> quite sure I see the issue here, I mean if something in the cinder base
> code breaks a driver it's still going to be broken, the only difference is
> that we'll actually know that it's broken.
>>  does a maintainer get to fix the issue before X happens as a result?
>> Does this happen every milestone I-1, I-2, I-3?  What happens if a
>> maintainer can't do it for every milestone for whatever reason?
> I don't think we've arrived at a point where we say X happens yet.  To
> start I'd view this as logging a bug if it doesn't pass.  Ultimately it
> would probably be interesting to consider things like removing a driver but
> there would have to be some process setup regarding how we try and address
> the issue before doing something drastic.
>> Just playing a bit of devil's advocate here.   I do like the idea though,
>> just depends on the "rules" setup and how it all applies when things don't
>> go well for a particular driver.
> Sure, that's fine and I think you bring up a good point.  The idea here is
> NOT to make things difficult or to try and keep drivers out etc, the idea
> is to release a better product.  There are a number of folks that have
> drivers that are "believed" to work but since there's no formal
> testing/integration it's just an assumption in the community.  This
> proposal would at least make it public information regarding whether a
> driver actually works or not.  I mean really, I don't think this is asking
> too much considering we require any patch to the projects in OpenStack to
> run these tests against the LVM driver to make sure things work.  This
> really isn't any different, except we don't *require* it for every check
> in.  We just do checks to make sure drivers are actually doing what we
> expect and end-users aren't surprised to find their driver doesn't actually
> work.
>> Cheers,
>> Walt
>>   Hey Everyone,
>>  Something I've been kicking around for quite a while now but never
>> really been able to get around to is the idea of requiring that drivers in
>> Cinder run a qualification test and submit results prior to introduction in
>> to Cinder.
>>  To elaborate a bit, the idea could start as something really simple
>> like the following:
>> 1. We'd add a functional_qual option/script to devstack
>>  2. Driver maintainer runs this script to setup devstack and configure
>> it to use their backend device on their own system.
>>  3. Script does the usual devstack install/configure and runs the volume
>> pieces of the Tempest gate tests.
>>  4. Grabs output and checksums of the directories in the devstack and
>> /opt/stack directories, bundles up the results for submission
>>  5. Maintainer submits results
>>  So why would we do this you ask?  Cinder is pretty heavy on the third
>> party driver plugin model which is fantastic.  On the other hand while
>> there are a lot of folks who do great reviews that catch things like syntax
>> or logic errors in the code, and unit tests do a reasonable job of
>> exercising the code it's difficult for folks to truly verify these devices
>> all work.
>>  I think it would be a very useful tool for initial introduction of a
>> new driver and even perhaps some sort of check that's run and submitted
>> again prior to milestone releases.
>>  This would also drive some more activity and contribution in to Tempest
>> with respect to getting folks like myself motivated to contribute more
>> tests (particularly in terms of new functionality) in to Tempest.
>>  I'd be interested to hear if folks have any interest or strong opinions
>> on this (positive or otherwise).  I know that some vendors like RedHat have
>> this sort of thing in place for certifications, and to be honest that
>> observation is something that caused me to start thinking about this again.
>>  There are a lot of gaps here regarding how the submission process would
>> look, but we could start relatively simple and grow from there if it's
>> valuable or just abandon the idea if it proves to be unpopular and a waste
>> of time.
>>  Anyway, I'd love to get feed-back from folks and see what they think.
>>  Thanks,
>>  John
>> _______________________________________________
>> OpenStack-dev mailing listOpenStack-dev at lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130730/8c444fdf/attachment.html>

More information about the OpenStack-dev mailing list