Open Stack

On 12/20/2013 03:43 PM, John Griffith wrote:
> Hey Everyone,
>
> So we merged the super simple driver cert test script in to devstack a
> while back.  For those that aren't familiar you can check it out here
> [1].  First iteration of this is simply a do it yourself config and
> run that goes through the same volume-tests that every cinder patch
> runs through the gate.
>
> There's definitely room for growth here but this seems like a
> reasonable first step.  The remaining question here is how do we want
> to use this?  I've made a coupe of suggestions that I'd like to review
> and get some feed-back.  To be clear this can obviously evolve over
> time, but I'd like to start somewhat simple, try it out and build off
> of if depending on how things go.  So with that here's a couple of
> options I've been considering:
>
> 1. File a bug in launchpad:
> This bug would be for tracking purposes, it would be something like
> "no cert results available for driver-X".  This would require that the
> driver maintainer download/install devstack, configure their driver
> and backend and then run the supplied script.
>
> The next question is what to do with the results, there are some options here:
>    a. Take the resultant tgz file and post it into the bug report as an
> attachment.  Assuming everything passes the bug can then be marked as
> closed/resolved.
>    b. Create a repo (or even a directory in the Cinder tree) that
> includes results files.  That way the bug is logged and a gerrit
> commit referencing the bug id is submitted and reviewed very similar
> to how we handle source changes.
>
> Option 'a' is path of least resistance, however it becomes a very
> manual process and it's somewhat ugly.  Option 'b' fits more with how
> we operate anyway, and provides some automation, and it also leaves a
> record of the cert process in the tree that makes visibility and
> tracking much easier.
I tend to like option B here.   Then as versions of Cinder ship, the 
'certification' files can go
along with it.

>
>
>
> 2.  Create a web/wiki page specifically for this information:
> This would basicly be a matrix of the drivers, and the current status
> of the cert results for the current iteration.  It would be something
> like a row for every driver in the tree and a column for "last cycle"
> and "current cycle".  We'd basically set it up so that the
> "current-cycle" entries are all listed as "not submitted" after the
> milestone is cut.  The current entries in that column would roll back
> to the "last-cycle" column.  Then the driver maintainer could
> run/update the matrix at any time during that cycle.
>
> The only thing with this is again it's very manual in terms of
> tracking, might be a bit confusing (this may make perfect sense to me,
> but seems like jibberish to others :)), and we'd want to have a
> repository to store the results files for people to reference.
I think this is a good idea as a reference point for users, but I tend to
like the automated mechanism of Option 1b.  I see option 1 and 2 being
parallel streams of information.  Option 1 being for the cinder
developer team and option 2 being for end users as a quick check to see
what drivers have passed the certification.

I think the general idea of certifying every driver on every milestone is a
good idea, especially since there are several vendors who are running
against master.  This mechanism could give them some assurances that
at least the drivers in Cinder are good to go, or are pending 
"certification".


Walt

>
> I'm open to ideas/suggestions here keeping in mind that the initial
> purpose is to provide publicly viewable information as to the
> integration status of the drivers in the Cinder project.  This would
> help people building OpenStack clouds to make sure that the backend
> devices they may be choosing actually implement all of the base
> features and that they actually work.  Vendors can of course choose
> not to participate, that just tells consumers "beware, vendor-a
> doesn't necessarily care all that much, or doesn't have time to test
> this".
>
> Anyway, hopefully this makes sense, if more clarification is needed I
> can try and clean up my descriptions a bit.
>
> Thanks,
> John
>
> [1]: https://github.com/openstack-dev/devstack/blob/master/driver_certs/cinder_driver_cert.sh
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> .
>