Open Stack

On Fri, Mar 31, 2023 at 11:26 AM Ghanshyam Mann <gmann at ghanshyammann.com>
wrote:

>  ---- On Wed, 29 Mar 2023 19:20:43 -0700  Takashi Kajinami  wrote ---
>  > Hello,
>  >
>  > I have had some local discussions with gmann, but I'd really like to
> move this discussion forwardto fix the broken stable/xena gate in heat so I
> will start this thread, hoping the thread can providemore context behind my
> proposal.
>  > Historically stable branches of heat have been frequently affected by
> any change in requirementsof tempest. This is mainly because in our CI we
> install our own in-tree integration tests[1] intotempest venv where tempest
> and heat-tempest-plugin are installed. Because in-tree integration testsare
> tied to that specific stable branch, this has been often causing conflicts
> in requirements(master constraint vs stable/X constraint).
>
> Let me explain the issue here. It is not because of using tempest from
> upper constraints or branchless things,
> it is because we are not setting the tempest venv constraints correctly
> for the target tempest run in the grenade job.
>
> We fixed the tempest venv constraints setting for the tempest test run on
> the base branch[1] but forgot to do the same
> for the target branch test run. As we do not have any grenade job except
> heat which is running tempest on the target branch
> in the grenade job, we could not face this issue and heat testing unhide
> it. I then reproduce it on a normal grenade job by
> running the tempest on target and the same issue[2][3].
>
> The issue is when base and target branches have different Tempest and
> constraints to use (for example, stable/wallaby uses old tempest
> and stable/wallaby constraints, but stable/xena use tempest master and
> master constraints); in such cases, we need to set proper constraints
> defined in devstack and then run tempest. It will happen in the grenade
> job running on the immediately supported branch of the latest EM.
>
This is the core problem in heat, which is conflicting what has been done
in heat testing.
During tests after upgrade we run not only tempest + heat-tempest-plugin
tests but also in-tree heat integration tests
which test more actual resources. However in-tree integration tests are
dependent on a specific stable/requirement.
So when we run tests in stable/xena then we need stable/xena constraints
installed in venv, which means we need to
install tempest which is compatible with stable/xena uc, rather than master
tempest. For this sake, I'm asking for
adding requirements so that we can install tempest with stable/xena u-c.
(Currently we do not explicitly install tempest
but it is installed as a dependency of heat-tempest-plugin. I tried to set
an explicit tempest version but it has been failing
for some reason.)

https://review.opendev.org/c/openstack/heat/+/878610

If running tempest tests after upgrade is not commonly done then we
probably can replace tempest by more simple
ones as is done for the core services such as keystone, or at least we can
get rid of integration tests. Though we still
likely face an issue with our normal integration tests which run the same
set of tests.



> I have pushed the grenade fix[4] and testing it by applying the same in
> heat[5]. If it work then I will push heat change
> form master itself and backported till stable/xena, so we fix it for all
> future EM/stable branches.
>
> [1] https://review.opendev.org/q/topic:bug%252F2003993
> [2] https://review.opendev.org/c/openstack/grenade/+/878247/1
> [3]
> https://zuul.opendev.org/t/openstack/build/1b503d359717459c9c77010608068e27/log/controller/logs/grenade.sh_log.txt#17184
> [4] https://review.opendev.org/c/openstack/grenade/+/879113
> [5] https://review.opendev.org/c/openstack/heat/+/872055
>
>
> -gmann
>
>  >
>  > [1]https://github.com/openstack/heat/tree/master/heat_integrationtests
>  > In the past we changed our test installation[2] to use stable
> constraint to avoid this conflicts,but this approach does no longer work
> since stable/xena because
>  > 1. stable/xena u-c no longer includes tempest
>  > 2. latest tempest CAN'T be installed with stable/xena u-c because
> current tempest requires    fasteners>=0.16.0 which conflicts with 0.14.1
> in stable/xena u-c.
>  > [2]
> https://review.opendev.org/c/openstack/heat/+/803890https://review.opendev.org/c/openstack/heat/+/848215
>  > I've proposed the change to pin tempest[3] in stable/xena u-c so that
> people can install tempestwith stable/xena u-c.
>  > [3] https://review.opendev.org/c/openstack/requirements/+/878228
>  > I understand the reason tempest was removed from u-c was that we should
> use the latest tempestto test recent stable releases.I agree we can keep
> tempest excluded for stable/yoga and onwardsbecause tempest is installable
> with their u-c, but stable/xena u-c is no longer compatible with
> master.Adding pin to xena u-c does not mainly affect the policy to test
> stable branches with latest tempestbecause for that we anyway need to use
> more recent u-c.
>  > I'm still trying to find out the workaround within heat but IMO adding
> tempest pin to stable/xena u-cis harmless but beneficial in case anyone is
> trying to use tempest with stable/xena u-c.
>  >
>  > Thank you,
>  > Takashi Kajinami
>  >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20230331/8d6270f3/attachment.htm>