[release][PTL] broken gates and stuck releases
Hello PTLs, For your information our gates are currently broken so we are not able to merge patches and so to release deliverables. indeed openstack/release rely on openstack/governance to validate releases. The governance repo relies on pybot2 which is no longer maintained and became not installable a few hours ago. That broke the release gate and led us to a catch-22 situation where we are not able to release the governance repo to fix our repo. Our current solution is to pull openstack/governance from git source instead of from pypi. By doing that we will pull the fixed version from opendev git and we will unblock our gates. In this way, we will be able to restart to merge patches and release deliverables. We currently wait for some feedback from the infra team to help us to force merging the needed fixes by using their gerrit super admin rights, hence, bypassing the catch-22 situation. This situation highlighted the cross dependency between these two repos (governance and releases) and I don't think that this is something healthy. I propose to keep pulling the governance repo from git source even after we will fix the catch-22 situation. Any opinions? Thanks for your understanding, we will inform you when we will have updates. https://opendev.org/openstack/governance/commit/9128e502253db9aac346ce25726a... https://review.opendev.org/c/openstack/releases/+/809626 -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/ https://twitter.com/4383hberaud
On 2021-09-17 12:41:17 +0200 (+0200), Herve Beraud wrote: [...]
Our current solution is to pull openstack/governance from git source instead of from pypi. By doing that we will pull the fixed version from opendev git and we will unblock our gates. In this way, we will be able to restart to merge patches and release deliverables.
We currently wait for some feedback from the infra team to help us to force merging the needed fixes by using their gerrit super admin rights, hence, bypassing the catch-22 situation.
By the time I was around, a solution was found which involved merely adjusting the release project requirements list, so no action was needed outside normal code review workflow.
This situation highlighted the cross dependency between these two repos (governance and releases) and I don't think that this is something healthy. I propose to keep pulling the governance repo from git source even after we will fix the catch-22 situation. Any opinions? [...]
The "proper" way to do this is to declare openstack/governance in the required-projects lists for the jobs which are using it, and then the tox role from zuul-jobs will know to consume it from locally supplied source instead of fetching it from PyPI. This also allows Zuul to handle cross-repository dependencies properly, so a requirements change would be able to successfully Depends-On some proposed change in governance even before it's merged. -- Jeremy Stanley
---- On Fri, 17 Sep 2021 08:51:54 -0500 Jeremy Stanley <fungi@yuggoth.org> wrote ----
On 2021-09-17 12:41:17 +0200 (+0200), Herve Beraud wrote: [...]
Our current solution is to pull openstack/governance from git source instead of from pypi. By doing that we will pull the fixed version from opendev git and we will unblock our gates. In this way, we will be able to restart to merge patches and release deliverables.
We currently wait for some feedback from the infra team to help us to force merging the needed fixes by using their gerrit super admin rights, hence, bypassing the catch-22 situation.
By the time I was around, a solution was found which involved merely adjusting the release project requirements list, so no action was needed outside normal code review workflow.
This situation highlighted the cross dependency between these two repos (governance and releases) and I don't think that this is something healthy. I propose to keep pulling the governance repo from git source even after we will fix the catch-22 situation. Any opinions? [...]
The "proper" way to do this is to declare openstack/governance in the required-projects lists for the jobs which are using it, and then the tox role from zuul-jobs will know to consume it from locally supplied source instead of fetching it from PyPI. This also allows Zuul to handle cross-repository dependencies properly, so a requirements change would be able to successfully Depends-On some proposed change in governance even before it's merged.
+1, and using data directly from governance repo make more sense than pip. I thought we release governance only for election scripts. can we remove governance release dependencies also, I have not checked those election scripts yet? and after that, we stop releasing governance itself and everyone can use it from the source. -gmann
-- Jeremy Stanley
Just a heads up to inform everyone that the situation is now fixed and our gates are now ok. Deliverables can be now released. Le ven. 17 sept. 2021 à 16:33, Ghanshyam Mann <gmann@ghanshyammann.com> a écrit :
---- On Fri, 17 Sep 2021 08:51:54 -0500 Jeremy Stanley <fungi@yuggoth.org> wrote ----
On 2021-09-17 12:41:17 +0200 (+0200), Herve Beraud wrote: [...]
Our current solution is to pull openstack/governance from git source instead of from pypi. By doing that we will pull the fixed version from opendev git and we will unblock our gates. In this way, we will be able to restart to merge patches and release deliverables.
We currently wait for some feedback from the infra team to help us to force merging the needed fixes by using their gerrit super admin rights, hence, bypassing the catch-22 situation.
By the time I was around, a solution was found which involved merely adjusting the release project requirements list, so no action was needed outside normal code review workflow.
This situation highlighted the cross dependency between these two repos (governance and releases) and I don't think that this is something healthy. I propose to keep pulling the governance repo from git source even after we will fix the catch-22 situation. Any opinions? [...]
The "proper" way to do this is to declare openstack/governance in the required-projects lists for the jobs which are using it, and then the tox role from zuul-jobs will know to consume it from locally supplied source instead of fetching it from PyPI. This also allows Zuul to handle cross-repository dependencies properly, so a requirements change would be able to successfully Depends-On some proposed change in governance even before it's merged.
+1, and using data directly from governance repo make more sense than pip. I thought we release governance only for election scripts. can we remove governance release dependencies also, I have not checked those election scripts yet? and after that, we stop releasing governance itself and everyone can use it from the source.
-gmann
-- Jeremy Stanley
-- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/ https://twitter.com/4383hberaud
participants (3)
-
Ghanshyam Mann
-
Herve Beraud
-
Jeremy Stanley