Open Stack

This may have already been raised/discussed, but I'm kinda confused so
thought I'd ask on the ML here.  The whole point of third party CI as I
recall was to run the same tests that we run in the official Gate against
third party drivers.  To me that would imply that a CI system/device that
marks itself as "GOOD" doesn't do things like add skips locally that aren't
in the tempest code already?

In other words, seems like cheating to say "My CI passes and all is good,
except for the tests that don't work which I skip... but pay no attention
to those please".

Did I miss something, isn't the whole point of Third Party CI to
demonstrate that a third parties backend is tested and functions to the
same degree that the reference implementations do? So the goal (using
Cinder for example) was to be able to say that any API call that works on
the LVM reference driver will work on the drivers listed in driverlog; and
that we know this because they run the same Tempest API tests?

Don't get me wrong, certainly not saying there's malice or things should be
marked as no good... but if the practice is to skip what you can't do then
maybe that should be documented in the driverlog submission, as opposed to
just stating "Yeah, we run CI successfully".

Thanks,
John
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150330/8e785fdb/attachment.html>