Open Stack

On Thu, Jan 16, 2014 at 6:28 AM, John Griffith
<john.griffith at solidfire.com> wrote:
> Hey Everyone,
>
> A while back I started talking about this idea of requiring Cinder
> driver contributors to run a super simple cert script (some info here:
> [1]).  Since then I've been playing with introduction of a third party
> gate check here in my own lab.  My proposal was to have a non-voting
> check that basically duplicates the base devstack gate test in my lab,
> but uses different back-end devices that I have available configured
> in Cinder to run periodic tests against.  Long term I'd like to be
> able to purpose this gear to also do something "more useful" for the
> over all OpenStack gating effort but to start it's strictly an
> automated verification of my Cinder driver/backend.
>
> What I'm questioning is how to report this information and the
> results.  Currently patches and reviews are our mechanism for
> triggering tests and providing feedback.  Myself and many other
> vendors that might like to participate in something like this
> obviously don't have the infrastructure to try and run something like
> this on every single commit.  Also since it would be non-voting it's
> difficult to capture and track the results.
>
> One idea that I had was to set something like what I've described
> above to run locally on a periodic basis (weekly, nightly etc) and
> publish results to something like a "third party verification
> dashboard".  So the idea would be that results from various third
> party tests would all adhere to a certain set of criteria WRT what
> they do and what they report  and those results would be logged and
> tracked publicly for anybody in the OpenStack community to access and
> view?

My concern here is how to identify what patch broke the third party
thing. If you run this once a week, then there are possible hundreds
of patches which might be responsible. How do you identify which one
is the winner?

> Does this seem like something that others would be interested in
> participating in?  I think it's extremely valuable for projects like
> Cinder that have dozens of backend devices, and regardless of other
> interest or participation in the community I intend to implement
> something like this on my own regardless.  It would just be
> interesting to see if we could have an organized and official effort
> to gather this sort of information and run these types of tests.
>
> Open to suggestions and thoughts as well as any of you that may
> already be doing this sort of thing.  By the way, I've been looking at
> things like SmokeStack and other third party gating checks to get some
> ideas as well.

Michael

-- 
Rackspace Australia