[openstack-dev] False Positive testing for 3rd party CI

Armando M. armamig at gmail.com
Fri Feb 21 21:08:18 UTC 2014


Nice one!

On 21 February 2014 11:22, Aaron Rosen <aaronorosen at gmail.com> wrote:
> This should fix the false positive for brocade:
> https://review.openstack.org/#/c/75486/
>
> Aaron
>
>
> On Fri, Feb 21, 2014 at 10:34 AM, Aaron Rosen <aaronorosen at gmail.com> wrote:
>>
>> Hi,
>>
>> Yesterday, I pushed a patch to review and was surprised that several of
>> the third party CI systems reported back that the patch-set worked where it
>> definitely shouldn't have. Anyways, I tested out my theory a little more and
>> it turns out a few of the 3rd party CI systems for neutron are just
>> returning  SUCCESS even if the patch set didn't run successfully
>> (https://review.openstack.org/#/c/75304/).
>>
>> Here's a short summery of what I found.
>>
>> Hyper-V CI -- This seems like an easy fix as it's posting "build
>> succeeded" but also puts to the side "test run failed". Would probably be a
>> good idea to remove the "build succeeded" message to avoid any confusion.
>>
>>
>> Brocade CI - From the log files it posts it shows that it tries to apply
>> my patch but fails:
>>
>> 2014-02-20 20:23:48 + cd /opt/stack/neutron
>> 2014-02-20 20:23:48 + git fetch
>> https://review.openstack.org/openstack/neutron.git refs/changes/04/75304/1
>> 2014-02-20 20:24:00 From https://review.openstack.org/openstack/neutron
>> 2014-02-20 20:24:00  * branch            refs/changes/04/75304/1 ->
>> FETCH_HEAD
>> 2014-02-20 20:24:00 + git checkout FETCH_HEAD
>> 2014-02-20 20:24:00 error: Your local changes to the following files would
>> be overwritten by checkout:
>> 2014-02-20 20:24:00 	etc/neutron/plugins/ml2/ml2_conf_brocade.ini
>> 2014-02-20 20:24:00
>> 	neutron/plugins/ml2/drivers/brocade/mechanism_brocade.py
>> 2014-02-20 20:24:00 Please, commit your changes or stash them before you
>> can switch branches.
>> 2014-02-20 20:24:00 Aborting
>> 2014-02-20 20:24:00 + cd /opt/stack/neutron
>>
>> but still continues running (without my patchset) and reports success. --
>> This actually looks like a devstack bug  (i'll check it out).
>>
>> PLUMgrid CI - Seems to always vote +1 without a failure
>> (https://review.openstack.org/#/dashboard/10117) though the logs are private
>> so we can't really tell whats going on.
>>
>> I was thinking it might be worth while or helpful to have a job that tests
>> that CI is actually fails when we expect it to.
>>
>> Best,
>>
>> Aaron
>>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



More information about the OpenStack-dev mailing list