[all][designate][manila] Grenade job failures caused by GLOBAL_VENV=True
Just in case you are not aware of the situation. The change to enable GLOBAL_VENV in grenade job was recently merged[1] in core grenade. As we saw when the same was introduced in devstack some time ago, some project got their (grenade) jobs broken and need a few adjustments. [1] https://review.opendev.org/c/openstack/grenade/+/930507 So far I'm aware that the change affected manila and designate, which I'm fixing by the changes below. Please be careful in case you see grenade job failing in your project. designate: - https://review.opendev.org/c/openstack/designate/+/932697 Disable GLOBAL_VENV in grenade This is temporal workaround and needs to be reverted once the actual fix below is merged in master and backported to stable 2024.1/2 - https://review.opendev.org/c/openstack/designate/+/932697 devstack: Fix manage.py execution with GLOBAL_VENV This actually fixes the current issue caused by GLOBAL_VENV=True. As noted above, we have to backport this to stable branches manila: - https://review.opendev.org/c/openstack/manila/+/932671 Disable GLOBAL_VENV in grenade This is temporal workaround and needs to be reverted once the actual fix below is merged in master and backported to stable 2024.1/2 - https://review.opendev.org/c/openstack/manila/+/932665 devstack: Fix missing manila command This actually fixes the current issue caused by GLOBAL_VENV=True. As noted above, we have to backport this to stable branches Thank you, Takashi -- Takashi Kajinami irc: tkajinam github: https://github.com/kajinamit launchpad: https://launchpad.net/~kajinamit
(Designate: "devstack: Fix manage.py execution with GLOBAL_VENV" correct link) https://review.opendev.org/c/openstack/designate/+/932688 On Mon, Oct 21, 2024 at 12:08 PM Takashi Kajinami <kajinamit@oss.nttdata.com> wrote:
Just in case you are not aware of the situation.
The change to enable GLOBAL_VENV in grenade job was recently merged[1] in core grenade. As we saw when the same was introduced in devstack some time ago, some project got their (grenade) jobs broken and need a few adjustments.
[1] https://review.opendev.org/c/openstack/grenade/+/930507
So far I'm aware that the change affected manila and designate, which I'm fixing by the changes below. Please be careful in case you see grenade job failing in your project.
designate: - https://review.opendev.org/c/openstack/designate/+/932697 Disable GLOBAL_VENV in grenade This is temporal workaround and needs to be reverted once the actual fix below is merged in master and backported to stable 2024.1/2
- https://review.opendev.org/c/openstack/designate/+/932697 devstack: Fix manage.py execution with GLOBAL_VENV This actually fixes the current issue caused by GLOBAL_VENV=True. As noted above, we have to backport this to stable branches
manila:
- https://review.opendev.org/c/openstack/manila/+/932671 Disable GLOBAL_VENV in grenade This is temporal workaround and needs to be reverted once the actual fix below is merged in master and backported to stable 2024.1/2
- https://review.opendev.org/c/openstack/manila/+/932665 devstack: Fix missing manila command This actually fixes the current issue caused by GLOBAL_VENV=True. As noted above, we have to backport this to stable branches
Thank you, Takashi
-- Takashi Kajinami irc: tkajinam github: https://github.com/kajinamit launchpad: https://launchpad.net/~kajinamit
On Mon, Oct 21, 2024, at 3:07 AM, Takashi Kajinami wrote:
Just in case you are not aware of the situation.
The change to enable GLOBAL_VENV in grenade job was recently merged[1] in core grenade. As we saw when the same was introduced in devstack some time ago, some project got their (grenade) jobs broken and need a few adjustments.
[1] https://review.opendev.org/c/openstack/grenade/+/930507
So far I'm aware that the change affected manila and designate, which I'm fixing by the changes below. Please be careful in case you see grenade job failing in your project.
designate: - https://review.opendev.org/c/openstack/designate/+/932697 Disable GLOBAL_VENV in grenade This is temporal workaround and needs to be reverted once the actual fix below is merged in master and backported to stable 2024.1/2
This change should not be required. Instead, fixes to Grenade should go into the oldest stable branch that requires the fix then port forward towards the master branch. I know this is not typical process but it enables you to ensure that everything is working (via depends-on among the changes) before anything merges and avoids unnecessary disable/enable changes.
- https://review.opendev.org/c/openstack/designate/+/932697 devstack: Fix manage.py execution with GLOBAL_VENV This actually fixes the current issue caused by GLOBAL_VENV=True. As noted above, we have to backport this to stable branches
manila:
- https://review.opendev.org/c/openstack/manila/+/932671 Disable GLOBAL_VENV in grenade This is temporal workaround and needs to be reverted once the actual fix below is merged in master and backported to stable 2024.1/2
- https://review.opendev.org/c/openstack/manila/+/932665 devstack: Fix missing manila command This actually fixes the current issue caused by GLOBAL_VENV=True. As noted above, we have to backport this to stable branches
This change symlinks a python command installed in the global devstack venv into /usr/local/bin. Can we avoid this entire class of problem by ensuring the global devstack venv bin dir is in $PATH instead? Then we don't need to micro manage each of these python commands and their presence in /usr/local/bin.
Thank you, Takashi
-- Takashi Kajinami irc: tkajinam github: https://github.com/kajinamit launchpad: https://launchpad.net/~kajinamit
On 10/21/24 22:01, Clark Boylan wrote:
On Mon, Oct 21, 2024, at 3:07 AM, Takashi Kajinami wrote:
Just in case you are not aware of the situation.
The change to enable GLOBAL_VENV in grenade job was recently merged[1] in core grenade. As we saw when the same was introduced in devstack some time ago, some project got their (grenade) jobs broken and need a few adjustments.
[1] https://review.opendev.org/c/openstack/grenade/+/930507
So far I'm aware that the change affected manila and designate, which I'm fixing by the changes below. Please be careful in case you see grenade job failing in your project.
designate: - https://review.opendev.org/c/openstack/designate/+/932697 Disable GLOBAL_VENV in grenade This is temporal workaround and needs to be reverted once the actual fix below is merged in master and backported to stable 2024.1/2
This change should not be required. Instead, fixes to Grenade should go into the oldest stable branch that requires the fix then port forward towards the master branch. I know this is not typical process but it enables you to ensure that everything is working (via depends-on among the changes) before anything merges and avoids unnecessary disable/enable changes.
I understand the suggested order works but I intended to avoid port-forwards based on some confusions caused by it in the past (eg master change is rebased which breaks reference by commit-id). Checking the solution by depends-on is a good idea so I'll try that early.
- https://review.opendev.org/c/openstack/designate/+/932697 devstack: Fix manage.py execution with GLOBAL_VENV This actually fixes the current issue caused by GLOBAL_VENV=True. As noted above, we have to backport this to stable branches
manila:
- https://review.opendev.org/c/openstack/manila/+/932671 Disable GLOBAL_VENV in grenade This is temporal workaround and needs to be reverted once the actual fix below is merged in master and backported to stable 2024.1/2
- https://review.opendev.org/c/openstack/manila/+/932665 devstack: Fix missing manila command This actually fixes the current issue caused by GLOBAL_VENV=True. As noted above, we have to backport this to stable branches
This change symlinks a python command installed in the global devstack venv into /usr/local/bin. Can we avoid this entire class of problem by ensuring the global devstack venv bin dir is in $PATH instead? Then we don't need to micro manage each of these python commands and their presence in /usr/local/bin.
The current "solution" is following what is already done in tempest. I tend to agree with adding the venv path to PATH but I'd like to be stick with the current way until we get a better solution merged in core devstack first. Note that the manila CLI is deprecated and we eventually have to replace it by openstack CLI. Because installation of openstack cli is taken care by devstack we may eventually get rid of the hack once the migration is done.
Thank you, Takashi
-- Takashi Kajinami irc: tkajinam github: https://github.com/kajinamit launchpad: https://launchpad.net/~kajinamit
---- On Mon, 21 Oct 2024 06:01:37 -0700 Clark Boylan wrote ---
On Mon, Oct 21, 2024, at 3:07 AM, Takashi Kajinami wrote:
Just in case you are not aware of the situation.
The change to enable GLOBAL_VENV in grenade job was recently merged[1] in core grenade. As we saw when the same was introduced in devstack some time ago, some project got their (grenade) jobs broken and need a few adjustments.
[1] https://review.opendev.org/c/openstack/grenade/+/930507
So far I'm aware that the change affected manila and designate, which I'm fixing by the changes below. Please be careful in case you see grenade job failing in your project.
designate: - https://review.opendev.org/c/openstack/designate/+/932697 Disable GLOBAL_VENV in grenade This is temporal workaround and needs to be reverted once the actual fix below is merged in master and backported to stable 2024.1/2
This change should not be required. Instead, fixes to Grenade should go into the oldest stable branch that requires the fix then port forward towards the master branch. I know this is not typical process but it enables you to ensure that everything is working (via depends-on among the changes) before anything merges and avoids unnecessary disable/enable changes.
Agree, we do not need to disable and re-enable it instead as you have fix up, we can directly merge the fix. it depends on the failure if we need to fix it in old branch or not. stable 2024.1/2 still have GLOBAL_VENV is still disabled so no change in their job and there should not be any failure there? Are you getting failure in stable 2024.1/2 grenade job also? -gmann
- https://review.opendev.org/c/openstack/designate/+/932697 devstack: Fix manage.py execution with GLOBAL_VENV This actually fixes the current issue caused by GLOBAL_VENV=True. As noted above, we have to backport this to stable branches
manila:
- https://review.opendev.org/c/openstack/manila/+/932671 Disable GLOBAL_VENV in grenade This is temporal workaround and needs to be reverted once the actual fix below is merged in master and backported to stable 2024.1/2
- https://review.opendev.org/c/openstack/manila/+/932665 devstack: Fix missing manila command This actually fixes the current issue caused by GLOBAL_VENV=True. As noted above, we have to backport this to stable branches
This change symlinks a python command installed in the global devstack venv into /usr/local/bin. Can we avoid this entire class of problem by ensuring the global devstack venv bin dir is in $PATH instead? Then we don't need to micro manage each of these python commands and their presence in /usr/local/bin.
Thank you, Takashi
-- Takashi Kajinami irc: tkajinam github: https://github.com/kajinamit launchpad: https://launchpad.net/~kajinamit
participants (4)
-
Clark Boylan
-
Ghanshyam Mann
-
Omer Schwartz
-
Takashi Kajinami