Open Stack

Hey Everyone,

A while back I started talking about this idea of requiring Cinder
driver contributors to run a super simple cert script (some info here:
[1]).  Since then I've been playing with introduction of a third party
gate check here in my own lab.  My proposal was to have a non-voting
check that basically duplicates the base devstack gate test in my lab,
but uses different back-end devices that I have available configured
in Cinder to run periodic tests against.  Long term I'd like to be
able to purpose this gear to also do something "more useful" for the
over all OpenStack gating effort but to start it's strictly an
automated verification of my Cinder driver/backend.

What I'm questioning is how to report this information and the
results.  Currently patches and reviews are our mechanism for
triggering tests and providing feedback.  Myself and many other
vendors that might like to participate in something like this
obviously don't have the infrastructure to try and run something like
this on every single commit.  Also since it would be non-voting it's
difficult to capture and track the results.

One idea that I had was to set something like what I've described
above to run locally on a periodic basis (weekly, nightly etc) and
publish results to something like a "third party verification
dashboard".  So the idea would be that results from various third
party tests would all adhere to a certain set of criteria WRT what
they do and what they report  and those results would be logged and
tracked publicly for anybody in the OpenStack community to access and
view?

Does this seem like something that others would be interested in
participating in?  I think it's extremely valuable for projects like
Cinder that have dozens of backend devices, and regardless of other
interest or participation in the community I intend to implement
something like this on my own regardless.  It would just be
interesting to see if we could have an organized and official effort
to gather this sort of information and run these types of tests.

Open to suggestions and thoughts as well as any of you that may
already be doing this sort of thing.  By the way, I've been looking at
things like SmokeStack and other third party gating checks to get some
ideas as well.

Thanks,
John