[openstack-dev] [tripleo] Blocking gate - do not recheck / rebase / approve any patch now (please)

Emilien Macchi emilien at redhat.com
Thu Oct 26 04:14:43 UTC 2017

On Wed, Oct 25, 2017 at 1:59 PM, Emilien Macchi <emilien at redhat.com> wrote:
> Quick update before being afk for some hours:
> - Still trying to land https://review.openstack.org/#/c/513701 (thanks
> Paul for promoting it in gate).


> - Disabling voting on scenario001 and scenario004 container jobs:
> https://review.openstack.org/#/c/515188/

Done, please be very careful while these jobs are not voting.
If any doubt, please ping me or fultonj or gfidente on #tripleo.

> - overcloudrc/keystone v2 workaround:
> https://review.openstack.org/#/c/515161/ (d0ugal will work on proper
> fix for https://bugs.launchpad.net/tripleo/+bug/1727454)

Merged - Dougal will work on the real fix this week but not urgent anymore.

> - Fixing zaqar/notification issues on
> https://review.openstack.org/#/c/515123 - we hope that helps to reduce
> some failures in gate

In gate right now and hopefully merged in less than 2 hours.
Otherwise, please keep rechecking it.
According to Thomas Hervé, il will reduce the change to timeout.

> - puppet-tripleo gate broken on stable branches (syntax jobs not
> running properly) - jeblair is looking at it now

jeblair will provide a fix hopefully this week but this is not
critical at this time.
Thanks Jim for your help.

> Once again, we'll need to retrospect and see why we reached that
> terrible state but let's focus on bringing our CI in a good shape
> again.
> Thanks a ton to everyone who is involved,

I'm now restoring all patches that I killed from the gate.
You can now recheck / rebase / approve what you want, but please save
our CI resources and do it with moderation. We are not done yet.

I won't call victory but we've merged almost all our blockers, one is
missing but currently in gate:
https://review.openstack.org/515123 - need babysit until merged.

Now let's see how RDO promotion works. We're close :-)

Thanks everyone,

> On Wed, Oct 25, 2017 at 7:25 AM, Emilien Macchi <emilien at redhat.com> wrote:
>> Status:
>> - Heat Convergence switch *might* be a reason why overcloud timeout so
>> much. Thomas proposed to disable it:
>> https://review.openstack.org/515077
>> - Every time a patch fails in the tripleo gate queue, it reset the
>> gate. I proposed to remove this common queue:
>> https://review.openstack.org/515070
>> - I cleared the patches in check and queue to make sure the 2 blockers
>> are tested and can be merged in priority. I'll keep an eye on it
>> today.
>> Any help is very welcome.
>> On Wed, Oct 25, 2017 at 5:58 AM, Emilien Macchi <emilien at redhat.com> wrote:
>>> We have been working very hard to get a package/container promotions
>>> (since 44 days) and now our blocker is
>>> https://review.openstack.org/#/c/513701/.
>>> Because the gate queue is huge, we decided to block the gate and kill
>>> all the jobs running there until we can get
>>> https://review.openstack.org/#/c/513701/ and its backport
>>> https://review.openstack.org/#/c/514584 (both are blocking the whole
>>> production chain).
>>> We hope to promote after these 2 patches, unless there is something
>>> else, in that case we would iterate to the next problem.
>>> We hope you understand and support us during this effort.
>>> So please do not recheck, rebase or approve any patch until further notice.
>>> Thank you,
>>> --
>>> Emilien Macchi
>> --
>> Emilien Macchi
> --
> Emilien Macchi

Emilien Macchi

More information about the OpenStack-dev mailing list