[openstack-dev] [tripleo] Gate is broken - Do not approve any patch until further notice

Bogdan Dobrelya bdobreli at redhat.com
Thu Aug 31 08:55:34 UTC 2017


On 31.08.2017 10:33, Michele Baldessari wrote:
> On Wed, Aug 30, 2017 at 11:31:14AM +0200, Bogdan Dobrelya wrote:
>> On 30.08.2017 6:54, Emilien Macchi wrote:
>>> On Tue, Aug 29, 2017 at 4:17 PM, Emilien Macchi <emilien at redhat.com> wrote:
>>>> We are currently dealing with 4 issues and until they are fix, please
>>>> do not approve any patch. We want to keep the gate clear to merge the
>>>> fixes for the 4 problems first.
>>>>
>>>> 1) devstack-gate broke us because we use it as a library (bad)
>>>> https://bugs.launchpad.net/tripleo/+bug/1713868
>>>>
>>>> 2) https://review.openstack.org/#/c/474578/ broke us and we're
>>>> reverting it https://bugs.launchpad.net/tripleo/+bug/1713832
>>>>
>>>> 3) We shouldn't build images on multinode jobs
>>>> https://bugs.launchpad.net/tripleo/+bug/1713167
>>>>
>>>> 4) We should use pip instead of git for delorean
>>>> https://bugs.launchpad.net/tripleo/+bug/1708832
>>>>
>>>>
>>>> Until further notice from Alex or myself, please do not approve any patch.
>>>
>>> The 4 problems have been mitigated.
>>> You can now proceed to normal review.
>>>
>>> Please do not recheck a patch without an elastic-recheck comment, we
>>> need to track all issues related to CI from now.
>>> Paul Belanger has been doing extremely useful work to help us, now
>>> let's use elastic-recheck more and stop blind rechecks.
>>> All known issues are in http://status.openstack.org/elastic-recheck/
>>> If one is missing, you're welcome to contribute by sending a patch to
>>> elastic-recheck. Example with https://review.openstack.org/#/c/498954/
>>
>> That's a great example! Let me follow up on that and share my beginner's
>> experience as well.
>>
>> Let's help with improving elastic-recheck queries to identify those
>> unknown or new failures, this is really important. This also trains
>> domain knowledge for particular areas, either openstack or *-infra, or
>> tripleo specific.
>>
>> As beginners, we could start with watching for failing tripleo-ci
>> periodic [0],[1] (available as RSS feeds) and gate jobs without e-r
>> comments, also from that page [2].
>>
>> Then fetching the logs locally with tools like getthelogs [3], or
>> looking into the logs.openstack.org directly, if advanced beginners wish so.
>>
>> Finally, identifying discovered (just do some grep, like I do with my
>> tool [4]) errorish patterns and helping with root cause analysis. And,
>> ideally, submitting new e-r queries (see also [5]) and corresponding lp
>> bugs. And absolutely ideally, help with addressing those as well. This
>> might be hard though as we may be not experts in some of the areas. Some
>> of the error messages would literally mean nothing to us. For me, the
>> most  But as the best effort, we could invite the right persons to
>> look into that, or at least ask folks on #tripleo or #openstack-infra.
>>
>> [0]
>> http://status.openstack.org/openstack-health/#/g/project/openstack-infra~2Ftripleo-ci
>> [1]
>> http://status.openstack.org/openstack-health/#/g/project/openstack~2Ftripleo-quickstart
>> [2] http://status.openstack.org/elastic-recheck/data/others.html
>> [3] https://review.openstack.org/#/c/492178/
>> [4] https://github.com/bogdando/fuel-log-parse/blob/master/fuel-log-parse.sh
>> [5]
>> https://docs.openstack.org/infra/elastic-recheck/readme.html#running-queries-locally
> 
> Thanks Bogdan, this is very helpful. Do we have some docs/readme on [5].
> It is failing here with a bunch of 404, so I presume I am missing a
> proper elasticRecheck.conf file or some other settings?
> 
> I was basically trying to validate https://review.openstack.org/#/c/499516/ before
> submitting it.

There is install docs [0] :)
Although for my case, the following worked as well (given
VENV=${HOME}/.virtualenvs):

$ mkvirtualenv erqtest
$ pip install -r requirements.txt
$ python setup.py develop
$ ${VENV}/erqtest/bin/elastic-recheck-query queries/foo.yaml

[0] https://docs.openstack.org/infra/elastic-recheck/installation.html

> 
> Thanks,
> Michele
> 


-- 
Best regards,
Bogdan Dobrelya,
Irc #bogdando



More information about the OpenStack-dev mailing list