[openstack-dev] False Positive testing for 3rd party CI

Hemanth Ravi hemanthraviml at gmail.com
Sun Feb 23 22:21:22 UTC 2014


One Convergence CI did report the third party tests as failed, but is
configured not to vote. Should we turn on the voting?

Thanks,
-hemanth


On Fri, Feb 21, 2014 at 1:29 PM, Carl Baldwin <carl at ecbaldwin.net> wrote:

> Aaron,
>
> I was thinking the same thing recently with this patch [1].  Patch
> sets 1-5 should have failed for any plugin besides ml2 yet some passed
> and I wondered how that could happen.  Kudos to those patches that
> failed my patch sets correctly.
>
> Carl
>
> [1] https://review.openstack.org/#/c/72565/
>
> On Fri, Feb 21, 2014 at 11:34 AM, Aaron Rosen <aaronorosen at gmail.com>
> wrote:
> > Hi,
> >
> > Yesterday, I pushed a patch to review and was surprised that several of
> the
> > third party CI systems reported back that the patch-set worked where it
> > definitely shouldn't have. Anyways, I tested out my theory a little more
> and
> > it turns out a few of the 3rd party CI systems for neutron are just
> > returning  SUCCESS even if the patch set didn't run successfully
> > (https://review.openstack.org/#/c/75304/).
> >
> > Here's a short summery of what I found.
> >
> > Hyper-V CI -- This seems like an easy fix as it's posting "build
> succeeded"
> > but also puts to the side "test run failed". Would probably be a good
> idea
> > to remove the "build succeeded" message to avoid any confusion.
> >
> >
> > Brocade CI - From the log files it posts it shows that it tries to apply
> my
> > patch but fails:
> >
> > 2014-02-20 20:23:48 + cd /opt/stack/neutron
> > 2014-02-20 20:23:48 + git fetch
> > https://review.openstack.org/openstack/neutron.gitrefs/changes/04/75304/1
> > 2014-02-20 20:24:00 From https://review.openstack.org/openstack/neutron
> > 2014-02-20 20:24:00  * branch            refs/changes/04/75304/1 ->
> > FETCH_HEAD
> > 2014-02-20 20:24:00 + git checkout FETCH_HEAD
> > 2014-02-20 20:24:00 error: Your local changes to the following files
> would
> > be overwritten by checkout:
> > 2014-02-20 20:24:00   etc/neutron/plugins/ml2/ml2_conf_brocade.ini
> > 2014-02-20 20:24:00
> >       neutron/plugins/ml2/drivers/brocade/mechanism_brocade.py
> > 2014-02-20 20:24:00 Please, commit your changes or stash them before you
> can
> > switch branches.
> > 2014-02-20 20:24:00 Aborting
> > 2014-02-20 20:24:00 + cd /opt/stack/neutron
> >
> > but still continues running (without my patchset) and reports success. --
> > This actually looks like a devstack bug  (i'll check it out).
> >
> > PLUMgrid CI - Seems to always vote +1 without a failure
> > (https://review.openstack.org/#/dashboard/10117) though the logs are
> private
> > so we can't really tell whats going on.
> >
> > I was thinking it might be worth while or helpful to have a job that
> tests
> > that CI is actually fails when we expect it to.
> >
> > Best,
> >
> > Aaron
> >
> >
> > _______________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140223/80f0e0b6/attachment.html>


More information about the OpenStack-dev mailing list