Open Stack

On Mon, Mar 30, 2015 at 6:21 PM, Rochelle Grober <rochelle.grober at huawei.com
> wrote:

>  Top posting… I believe the main issue was a problem with snapshots that
> caused false negatives for most cinder drivers.  But, that got fixed.
> Unfortunately, we haven’t yet established a good process to notify third
> parties when skipped tests are fixed and should be “unskipped”.  Maybe
> tagging the tests can help on this.  But, I really do think this round was
> a bit of first run gotchas and rookie mistakes on all sides.  A good post
> mortem on how to better communicate changes and deadlines may go a long way
> to smooth these out in the next round.
>
>
>
> --Rocky
>
>
>
> John Griffith on Monday, March 30, 2015 15:36 wrote:
>
>  On Mon, Mar 30, 2015 at 4:06 PM, Doug Wiegley <
> dougwig at parksidesoftware.com> wrote:
>
> A few reasons, I’m sure there are others:
>
>
>
> - Broken tests that hardcode something about the ref implementation. The
> test needs to be fixed, of course, but in the meantime, a constantly
> failing CI is worthless (hello, lbaas scenario test.)
>
> ​Certainly... but that's relatively easy to fix (bug/patch to Tempest).
> Although that's not actually the case in this particular context as there
> are a handful of third party devices that run the full set of tests that
> the ref driver runs with no additional skips or modifications.
>
> ​
>
>
>
>  - Test relies on some “optional” feature, like overlapping IP subnets
> that the backend doesn’t support.  I’d argue it’s another case of broken
> tests if they require an optional feature, but it still needs skipping in
> the meantime.
>
>
>
> ​This may be something specific to Neutron perhaps?  In Cinder LVM is
> pretty much the "lowest common denominator".  I'm not aware of any volume
> tests in Tempest that rely on optional features that don't pick this up
> automatically out of the config (like multi-backend for example).
>
> ​
>
>
>
>  - Some new feature added to an interface, in the presence of
> shims/decomposed drivers/plugins (e.g. adding TLS termination support to
> lbaas.) Those implementations will lag the feature commit, by definition.
>
>
>
> ​Yeah, certainly I think this highlights some of the differences between
> Cinder and Neutron perhaps and the differences in complexity.
>
> Thanks for the feedback... I don't disagree per say, however Cinder is set
> up a bit different here in terms of expectations for base functionality
> requirements and compatibility but your points are definitely well taken.
> ​
>
>
>
> Thanks,
>
> doug
>
>
>
>
>
>   On Mar 30, 2015, at 2:54 PM, John Griffith <john.griffith8 at gmail.com>
> wrote:
>
>
>
> This may have already been raised/discussed, but I'm kinda confused so
> thought I'd ask on the ML here.  The whole point of third party CI as I
> recall was to run the same tests that we run in the official Gate against
> third party drivers.  To me that would imply that a CI system/device that
> marks itself as "GOOD" doesn't do things like add skips locally that aren't
> in the tempest code already?
>
>
>
> In other words, seems like cheating to say "My CI passes and all is good,
> except for the tests that don't work which I skip... but pay no attention
> to those please".
>
>
>
> Did I miss something, isn't the whole point of Third Party CI to
> demonstrate that a third parties backend is tested and functions to the
> same degree that the reference implementations do? So the goal (using
> Cinder for example) was to be able to say that any API call that works on
> the LVM reference driver will work on the drivers listed in driverlog; and
> that we know this because they run the same Tempest API tests?
>
>
>
> Don't get me wrong, certainly not saying there's malice or things should
> be marked as no good... but if the practice is to skip what you can't do
> then maybe that should be documented in the driverlog submission, as
> opposed to just stating "Yeah, we run CI successfully".
>
>
>
> Thanks,
>
> John
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
​Not top posting...

>> I believe the main issue was a problem with snapshots that caused false
negatives for most cinder drivers.  But, that got fixed

​Huh?  What was the problem, where was the problem, who/what fixed it, was
there a bug logged somewhere, what comprises *most* Cinder drivers?

Not disputing what you say, but for me it's just raised more questions than
anything else.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150330/8591a309/attachment.html>