Open Stack

On Mon, Mar 30, 2015 at 7:26 PM, <Arkady_Kanevsky at dell.com> wrote:

> Another scenario.
>
> The default LVM driver is local to cinder service.  Thus, it may work fine
> as soon as you go outside controller node it does not.
>
> We had a discussion on choosing different default driver and expect that
> discussion to continue.
>
>
>
> Not all drivers support all features. We have a table that list which
> features each driver support.
>
>
>
> The question I would ask is setting which test to skip in the driver is
> the right place?
>
> Why not specify it in the Tempest which driver run against.
>
> Then we can setup rules when drivers should remove themselves from that
> blackout list.
>
> That is easier to track, can be cleanly used by defcore and for tagging.
>
>
>
> Thanks,
>
> Arkady
>
>
>
> *From:* John Griffith [mailto:john.griffith8 at gmail.com]
> *Sent:* Monday, March 30, 2015 8:12 PM
> *To:* OpenStack Development Mailing List (not for usage questions)
> *Subject:* Re: [openstack-dev] [OpenStack-Dev] [third-party-ci]
> Clarifications on the goal and skipping tests
>
>
>
>
>
>
>
> On Mon, Mar 30, 2015 at 6:21 PM, Rochelle Grober <
> rochelle.grober at huawei.com> wrote:
>
> Top posting… I believe the main issue was a problem with snapshots that
> caused false negatives for most cinder drivers.  But, that got fixed.
> Unfortunately, we haven’t yet established a good process to notify third
> parties when skipped tests are fixed and should be “unskipped”.  Maybe
> tagging the tests can help on this.  But, I really do think this round was
> a bit of first run gotchas and rookie mistakes on all sides.  A good post
> mortem on how to better communicate changes and deadlines may go a long way
> to smooth these out in the next round.
>
>
>
> --Rocky
>
>
>
> John Griffith on Monday, March 30, 2015 15:36 wrote:
>
> On Mon, Mar 30, 2015 at 4:06 PM, Doug Wiegley <
> dougwig at parksidesoftware.com> wrote:
>
> A few reasons, I’m sure there are others:
>
>
>
> - Broken tests that hardcode something about the ref implementation. The
> test needs to be fixed, of course, but in the meantime, a constantly
> failing CI is worthless (hello, lbaas scenario test.)
>
> ​Certainly... but that's relatively easy to fix (bug/patch to Tempest).
> Although that's not actually the case in this particular context as there
> are a handful of third party devices that run the full set of tests that
> the ref driver runs with no additional skips or modifications.
>
> ​
>
>
>
> - Test relies on some “optional” feature, like overlapping IP subnets that
> the backend doesn’t support.  I’d argue it’s another case of broken tests
> if they require an optional feature, but it still needs skipping in the
> meantime.
>
>
>
> ​This may be something specific to Neutron perhaps?  In Cinder LVM is
> pretty much the "lowest common denominator".  I'm not aware of any volume
> tests in Tempest that rely on optional features that don't pick this up
> automatically out of the config (like multi-backend for example).
>
> ​
>
>
>
> - Some new feature added to an interface, in the presence of
> shims/decomposed drivers/plugins (e.g. adding TLS termination support to
> lbaas.) Those implementations will lag the feature commit, by definition.
>
>
>
> ​Yeah, certainly I think this highlights some of the differences between
> Cinder and Neutron perhaps and the differences in complexity.
>
> Thanks for the feedback... I don't disagree per say, however Cinder is set
> up a bit different here in terms of expectations for base functionality
> requirements and compatibility but your points are definitely well taken.
> ​
>
>
>
> Thanks,
>
> doug
>
>
>
>
>
> On Mar 30, 2015, at 2:54 PM, John Griffith <john.griffith8 at gmail.com>
> wrote:
>
>
>
> This may have already been raised/discussed, but I'm kinda confused so
> thought I'd ask on the ML here.  The whole point of third party CI as I
> recall was to run the same tests that we run in the official Gate against
> third party drivers.  To me that would imply that a CI system/device that
> marks itself as "GOOD" doesn't do things like add skips locally that aren't
> in the tempest code already?
>
>
>
> In other words, seems like cheating to say "My CI passes and all is good,
> except for the tests that don't work which I skip... but pay no attention
> to those please".
>
>
>
> Did I miss something, isn't the whole point of Third Party CI to
> demonstrate that a third parties backend is tested and functions to the
> same degree that the reference implementations do? So the goal (using
> Cinder for example) was to be able to say that any API call that works on
> the LVM reference driver will work on the drivers listed in driverlog; and
> that we know this because they run the same Tempest API tests?
>
>
>
> Don't get me wrong, certainly not saying there's malice or things should
> be marked as no good... but if the practice is to skip what you can't do
> then maybe that should be documented in the driverlog submission, as
> opposed to just stating "Yeah, we run CI successfully".
>
>
>
> Thanks,
>
> John
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> ​Not top posting...
>
>
>
> >> I believe the main issue was a problem with snapshots that caused
> false negatives for most cinder drivers.  But, that got fixed
>
>
>
> ​Huh?  What was the problem, where was the problem, who/what fixed it,
> was there a bug logged somewhere, what comprises *most* Cinder drivers?
>
>
>
> Not disputing what you say, but for me it's just raised more questions
> than anything else.
>
>
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
​This has gone horribly wrong and kinda misses the point of my original
post/request.  The good thing is that it turns out Mike already had another
thread he started last week that I somehow missed.

Now back to your regularly scheduled mail list.

John​
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150330/60966b1b/attachment.html>