<div dir="ltr"><div class="gmail_default" style="font-family:courier new,monospace"><br></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Jul 30, 2013 at 11:47 AM, Walter A. Boring IV <span dir="ltr"><<a href="mailto:walter.boring@hp.com" target="_blank">walter.boring@hp.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>So how frequently would this be
required and when do the results need to be provided?<br></div></div></blockquote><div><br></div><div class="gmail_default" style="font-family:'courier new',monospace">To start I'd like to propose any new driver, and then each driver upon milestone release, or at the very least each cycle.</div>
<div class="gmail_default" style="font-family:'courier new',monospace"></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000">
<div>
I generally think this is a good idea, but as well all know
that the project seems to be in flux at times around the
milestones. Especially around G3 when we changed the way the CONF
object was accessed, almost all drivers failed to work right after
G3 when cinder.conf was configured with a multi-backend setup. It
took a week or so after G3 before everyone</div></div></blockquote><div><br></div><div class="gmail_default" style="font-family:'courier new',monospace">IMO this is exactly why something like this would actually be more beneficial.</div>
<div class="gmail_default" style="font-family:'courier new',monospace"></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000">
<div> was able to go through
drivers and clean them up and get them to work in this scenario.
I'm not trying to be a wet blanket on this idea, but I think we
just have to be careful. I have no problem running my drivers
through this and providing the results. What happens when
something changes in cinder that causes drivers to fail and the
test runs fail? How long</div></div></blockquote><div><br></div><div class="gmail_default" style="font-family:'courier new',monospace">We should be extra careful to keep this sort of thing from happening, but again without doing any sort of public testing/results here we run the risk of nobody knowing it's broken until a customer encounters it. I'm also not quite sure I see the issue here, I mean if something in the cinder base code breaks a driver it's still going to be broken, the only difference is that we'll actually know that it's broken.</div>
<div class="gmail_default" style="font-family:'courier new',monospace"></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000">
<div> does a maintainer get to fix the issue
before X happens as a result? Does this happen every milestone
I-1, I-2, I-3? What happens if a maintainer can't do it for every
milestone for whatever reason?<br></div></div></blockquote><div><br></div><div class="gmail_default" style="font-family:'courier new',monospace">I don't think we've arrived at a point where we say X happens yet. To start I'd view this as logging a bug if it doesn't pass. Ultimately it would probably be interesting to consider things like removing a driver but there would have to be some process setup regarding how we try and address the issue before doing something drastic.</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000"><div>
<br>
Just playing a bit of devil's advocate here. I do like the idea
though, just depends on the "rules" setup and how it all applies
when things don't go well for a particular driver.<br></div></div></blockquote><div><br></div><div class="gmail_default" style="font-family:'courier new',monospace">Sure, that's fine and I think you bring up a good point. The idea here is NOT to make things difficult or to try and keep drivers out etc, the idea is to release a better product. There are a number of folks that have drivers that are "believed" to work but since there's no formal testing/integration it's just an assumption in the community. This proposal would at least make it public information regarding whether a driver actually works or not. I mean really, I don't think this is asking too much considering we require any patch to the projects in OpenStack to run these tests against the LVM driver to make sure things work. This really isn't any different, except we don't *require* it for every check in. We just do checks to make sure drivers are actually doing what we expect and end-users aren't surprised to find their driver doesn't actually work.</div>
<div class="gmail_default" style="font-family:'courier new',monospace"></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000">
<div>
<br>
Cheers,<br>
Walt<br>
<br>
<br>
</div>
<blockquote type="cite"><div><div class="h5">
<div dir="ltr">
<div style="font-family:courier new,monospace">Hey Everyone,</div>
<div style="font-family:courier new,monospace"><br>
</div>
<div style="font-family:courier new,monospace">
Something I've been kicking around for quite a while now but
never really been able to get around to is the idea of
requiring that drivers in Cinder run a qualification test and
submit results prior to introduction in to Cinder.</div>
<div style="font-family:courier new,monospace"><br>
</div>
<div style="font-family:courier new,monospace">To elaborate a bit, the idea could start as
something really simple like the following:</div>
<div style="font-family:courier new,monospace">1. We'd add a functional_qual option/script to
devstack</div>
<div style="font-family:courier new,monospace"><br>
</div>
<div style="font-family:courier new,monospace">
2. Driver maintainer runs this script to setup devstack and
configure it to use their backend device on their own system.</div>
<div style="font-family:courier new,monospace"><br>
</div>
<div style="font-family:courier new,monospace">
3. Script does the usual devstack install/configure and runs
the volume pieces of the Tempest gate tests.</div>
<div style="font-family:courier new,monospace"><br>
</div>
<div style="font-family:courier new,monospace">
4. Grabs output and checksums of the directories in the
devstack and /opt/stack directories, bundles up the results
for submission</div>
<div style="font-family:courier new,monospace"><br>
</div>
<div style="font-family:courier new,monospace">
5. Maintainer submits results</div>
<div style="font-family:courier new,monospace"><br>
</div>
<div style="font-family:courier new,monospace">So why would we do this you ask? Cinder is
pretty heavy on the third party driver plugin model which is
fantastic. On the other hand while there are a lot of folks
who do great reviews that catch things like syntax or logic
errors in the code, and unit tests do a reasonable job of
exercising the code it's difficult for folks to truly verify
these devices all work.</div>
<div style="font-family:courier new,monospace"><br>
</div>
<div style="font-family:courier new,monospace">I think it would be a very useful tool for
initial introduction of a new driver and even perhaps some
sort of check that's run and submitted again prior to
milestone releases.</div>
<div style="font-family:courier new,monospace"><br>
</div>
<div style="font-family:courier new,monospace">This would also drive some more activity and
contribution in to Tempest with respect to getting folks like
myself motivated to contribute more tests (particularly in
terms of new functionality) in to Tempest.</div>
<div style="font-family:courier new,monospace"><br>
</div>
<div style="font-family:courier new,monospace">I'd be interested to hear if folks have any
interest or strong opinions on this (positive or otherwise).
I know that some vendors like RedHat have this sort of thing
in place for certifications, and to be honest that observation
is something that caused me to start thinking about this
again.</div>
<div style="font-family:courier new,monospace"><br>
</div>
<div style="font-family:courier new,monospace">There are a lot of gaps here regarding how the
submission process would look, but we could start relatively
simple and grow from there if it's valuable or just abandon
the idea if it proves to be unpopular and a waste of time.</div>
<div style="font-family:courier new,monospace"><br>
</div>
<div style="font-family:courier new,monospace">Anyway, I'd love to get feed-back from folks
and see what they think.</div>
<div style="font-family:courier new,monospace"><br>
</div>
<div style="font-family:courier new,monospace">Thanks,</div>
<div style="font-family:courier new,monospace">
John</div>
<div style="font-family:courier new,monospace"><br>
</div>
</div>
<br>
<fieldset></fieldset>
<br>
</div></div><div class="im"><pre>_______________________________________________
OpenStack-dev mailing list
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
</div></blockquote>
<br>
</div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div></div>