[openstack-dev] [nova] Mellanox CI Issues

John Garbutt john at johngarbutt.com
Tue Jun 9 10:12:21 UTC 2015


On 5 June 2015 at 15:29, Moshe Levi <moshele at mellanox.com> wrote:
> Hi Dan,
>
> I just want to update you that the report success after short less-than-a-minute run is Zuul problem  see [1].
> This is happened because we apply filter rules to run only on PCI code to reduce load on our CI System.

I think the Xen Project CI is having similar issues due to using Zuul
and skipping tests causing an empty positive vote.

> Unfortunately  we missed this issue  in our monitoring because all the job in the Jenkins were looking good and
> just Zuul reports also when no Jenkins job was scheduled.
>
> I talked also with John Garbutt and it seem that the recommendation is to filter only doc and test folders,
> but still I would like to filter some other folders like
> 1. nova/nova/virt/hyperv
> 2. nova/nova/virt/ironic
> 3. nova/nova/virt/vmwareapi
> 4. nova/nova/virt/xenapi
>
> Is there any Nova CI guidelines  for file filtering?

Not yet. Ideally we want you to run on every change, that gives us the
most confidence in the tests.

But skipping docs only stuff and unit test only stuff makes sense
though, just like how we don't run tempest for those changes.

Now that might not be possible due to hardware costs, or similar, at
which point, we should talk about what things it might make sense to
skip. Your above list makes quite a bit of sense I guess.

Thanks,
John


> Again I would like to apologize for the inconvenient.
>
> [1] https://review.openstack.org/#/c/188383/
>
> Thank,
>         Moshe Levi.
>
> -----Original Message-----
> From: Lenny Verkhovsky
> Sent: Monday, June 01, 2015 11:21 PM
> To: Dan Smith; OpenStack Development Mailing List (not for usage questions)
> Cc: Moshe Levi
> Subject: RE: [nova] Mellanox CI Issues
>
> Hi Dan,
> I disabled Mellanox Nova CI from commenting and will check this issue first thing tomorrow morning.
> Thanks and my deepest apologies.
>
> Lenny Verkhovsky
>
>
> -----Original Message-----
> From: Dan Smith [mailto:dms at danplanet.com]
> Sent: Monday, June 01, 2015 7:41 PM
> To: OpenStack Development Mailing List (not for usage questions)
> Cc: Moshe Levi; Lenny Verkhovsky
> Subject: [nova] Mellanox CI Issues
>
> Hi,
>
> Mellanox CI has been broken for a while now. Test runs are reported as "successful" after an impossibly short less-than-a-minute run. Could the owners of this please take a look and address the problem? At least disabling commenting while working on the issue would be helpful.
>
> Also, on success, the bot doesn't post the log files, which is (a) inconsistent with other test bots and (b) not very helpful for validating that success is real. This is especially relevant right now, given that we know the success reports are erroneous at the moment.
>
> This is the second time (in recent memory) that Mellanox CI has gone off the rails for a decent amount of time without being noticed by the owners. If this continues, I'll be in favor of removing commenting privileges for this account and will be hesitant to throw in my support for re-enabling it. Running CI against gerrit comes with serious responsibility for monitoring!
>
> Thanks!
>
> --Dan
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



More information about the OpenStack-dev mailing list