Open Stack

Hi,

The backported uwsgi fixes cannot merge yet, as Tempest jobs are 
failing. (As an ansible post_run role in Tempest started to fail, too, - 
in Rocky and older branches - after the new base images were introduced.)

Gmann prepared a workaround [1], unfortunately it does not work as 
expected (as far as I understand) and still needs modification in 
Tempest. Which is a catch 22, as the root of the issue is that Tempest 
cannot be modified (in Rocky, Queens, ...), as it does not have stable 
branches, just a master branch. But the stable branches in Extended 
Maintenance are pinned to an older tag from Tempest and we cannot modify 
that version of Tempest.

In my opinion, one solution could be to cut a (stable?) branch from 
Tempest that can be used for branches currently in Extended Maintenance 
(rocky, queens, ...) and fix there the failing role. I know that Tempest 
is branchless, but can we have an exception somehow, just to be able to 
fix the Rocky, Queens, Pike branches?

[1] https://review.opendev.org/#/q/topic:move-stackviz-role

Thanks,

Előd


On 2020. 06. 25. 12:03, Renat Akhmerov wrote:
> Hi, any updates on the older branches? Specifically, we’re interested 
> in queens. We have a patch [1] that waits for it.
>
> [1] https://review.opendev.org/#/c/736129/
>
> Thanks
>
> Renat Akhmerov
> @Nokia
> On 23 Jun 2020, 23:17 +0700, Ghanshyam Mann <gmann at ghanshyammann.com>, 
> wrote:
>> ---- On Wed, 17 Jun 2020 13:17:26 -0500 Ghanshyam Mann 
>> <gmann at ghanshyammann.com> wrote ----
>>> Hello Everyone,
>>>
>>> As you know devstack (so does grenade) got broken due to uwsgi new 
>>> release, the master branch is fixed
>>> and stable branches are in progress[1]. But It is hard to maintain 
>>> or fix the EM stable for those issues. Especially
>>> the greande job which depends on the source branch (previous branch 
>>> of one where the job is running).
>>> For example, for stein grenade job, we need to keep rocky branch 
>>> working and fix if failure.
>>>
>>> Till now, we have not removed the grenade testing from any of the EM 
>>> stable branches because they
>>> were running fine till now but with uwsgi issues, those are failing 
>>> and need more work to fix. This triggers
>>> the discussion of grenade testing on EM stable branches.
>>>
>>> Usual policy for grenade testing is to keep the job running from the 
>>> 'oldest supported stable +1' branch.
>>> For example, if stein is the oldest supported stable (in the old 
>>> stable definition) then run grenade from train onwards.
>>> But with the Extended Maintainance model, defining 'oldest supported 
>>> stable' is not clear whether it is the oldest
>>> non-EM(stein) or oldest EM stable(ocata).
>>>
>>> To make it clear, we discussed this in the QA channel and come up 
>>> with the below proposal.
>>>
>>> * 'oldest' is the oldest non-EM. In current time, it is stein.
>>> * With the above 'oldest' definition, we will:
>>> ** Make grenade jobs as n-v on all EM stable branches (which is till 
>>> stable/rocky as of today) + on stable/stein also because that is 
>>> 'oldest' as of today.
>>> ** Keep supporting and running grenade job on 'oldest+1' which is 
>>> stable/train onwards as of today.
>>>
>>> NOTE: we will make n-v when they start failing and anyone can 
>>> volunteer to fix them and change back to voting.
>>> elod expressed interest to work on current failure.
>>>
>>> If no objection to the above proposal, I will document it on the 
>>> grenade documents to follow it whenever we see EM failing and need 
>>> more work.
>>> In Tempest, we already have the EM stable testing policy documented 
>>> which is to support those till they run fine[2].
>>
>> I have pushed the patches making grenade jobs non-voting on 
>> stable/stein[1]. With that patch merge in your project, it will bring
>> back the stable/stein onwards gate green. Once '<= stable/rocky] gate 
>> is up you can backport these grenade-n-v patches.
>>
>> Also, I am documenting the process in grenade for future reference.
>> - https://review.opendev.org/#/c/736866/
>>
>> [1] 
>> https://review.opendev.org/#/q/topic:grenade-em-nv+(status:open+OR+status:merged)
>>
>> -gmann
>>
>>
>>>
>>> [1] 
>>> http://lists.openstack.org/pipermail/openstack-discuss/2020-June/015496.html
>>> [2] 
>>> https://docs.openstack.org/tempest/latest/stable_branch_support_policy.html
>>> [3] 
>>> http://eavesdrop.openstack.org/irclogs/%23openstack-qa/%23openstack-qa.2020-06-17.log.html#t2020-06-17T14:12:42
>>>
>>> -gmann
>>>
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20200625/d7d0e975/attachment-0001.html>