[TripleO] gate blocker - impacting all quickstart-based jobs - openstack-ansible-os_tempest
Hello All,
We have a check/gate blocker on all TripleO quickstart-based jobs, as described in:
https://bugs.launchpad.net/tripleo/+bug/1967430
[1] commit to openstack-ansible-os_tempest removed setup.py and is causing failings in all quickstart jobs.
A revert was proposed but will not be workable - we are waiting on another fix.
Please hold rechecks until this is resolved.
[1] https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/835969
Thank you!
On Fri, Apr 1, 2022 at 12:14 AM Ronelle Landy rlandy@redhat.com wrote:
Hello All,
We have a check/gate blocker on all TripleO quickstart-based jobs, as described in:
https://bugs.launchpad.net/tripleo/+bug/1967430
[1] commit to openstack-ansible-os_tempest removed setup.py and is causing failings in all quickstart jobs.
A revert was proposed but will not be workable - we are waiting on another fix.
Please hold rechecks until this is resolved.
[1] https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/835969
Unfortunately looks like the core group on that repo is empty [1]. I added some folks into CC here that merged the original patch. Folks can you please help us merge the fix at https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/8360...
TripleO gate is blocked until we merge ansible-role-python_venv_build/+/836091
please help :D
[1] https://review.opendev.org/admin/groups/3474fc86368161e5288be01295041a089a10...
Thank you!
Hey there!
I have quick question - do you think it's valid approach to install Ansible roles as python packages? This smells sooooo fishy since ansible-galaxy is a thing along with requirements.yml...
So actual question is - do you have any plans on changing this approach to more Ansible way anytime soon?
пт, 1 апр. 2022 г., 8:19 Marios Andreou marios@redhat.com:
On Fri, Apr 1, 2022 at 12:14 AM Ronelle Landy rlandy@redhat.com wrote:
Hello All,
We have a check/gate blocker on all TripleO quickstart-based jobs, as described in:
https://bugs.launchpad.net/tripleo/+bug/1967430
[1] commit to openstack-ansible-os_tempest removed setup.py and is causing failings in all quickstart jobs.
A revert was proposed but will not be workable - we are waiting on another fix.
Please hold rechecks until this is resolved.
[1] https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/835969
Unfortunately looks like the core group on that repo is empty [1]. I added some folks into CC here that merged the original patch. Folks can you please help us merge the fix at https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/8360...
TripleO gate is blocked until we merge ansible-role-python_venv_build/+/836091
please help :D
[1] https://review.opendev.org/admin/groups/3474fc86368161e5288be01295041a089a10...
Thank you!
On Sun, Apr 3, 2022 at 8:10 PM Dmitriy Rabotyagov noonedeadpunk@gmail.com wrote:
Hey there!
I have quick question - do you think it's valid approach to install Ansible roles as python packages? This smells sooooo fishy since ansible-galaxy is a thing along with requirements.yml...
So actual question is - do you have any plans on changing this approach to more Ansible way anytime soon?
Hi
yes agreed and it was discussed in the team... the focus was on unblocking the gate for now but what you suggest is being worked on there https://review.opendev.org/c/openstack/tripleo-quickstart/+/836104
regards
пт, 1 апр. 2022 г., 8:19 Marios Andreou marios@redhat.com:
On Fri, Apr 1, 2022 at 12:14 AM Ronelle Landy rlandy@redhat.com wrote:
Hello All,
We have a check/gate blocker on all TripleO quickstart-based jobs, as described in:
https://bugs.launchpad.net/tripleo/+bug/1967430
[1] commit to openstack-ansible-os_tempest removed setup.py and is causing failings in all quickstart jobs.
A revert was proposed but will not be workable - we are waiting on another fix.
Please hold rechecks until this is resolved.
[1] https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/835969
Unfortunately looks like the core group on that repo is empty [1]. I added some folks into CC here that merged the original patch. Folks can you please help us merge the fix at https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/8360...
TripleO gate is blocked until we merge ansible-role-python_venv_build/+/836091
please help :D
[1] https://review.opendev.org/admin/groups/3474fc86368161e5288be01295041a089a10...
Thank you!
Full disclosure: I have only surface level understanding of how ansible galaxy actually works on the inside. My exposure to it is rather limited and it's possible that all of my concerns have perfectly valid responses I'm not aware of. Furthermore, I do believe that we could utilize ansible galaxy a bit more than we do.
That being said, I do think that we should be cautious when changing the way we package and deliver. Even if everything works out we are possibly setting ourselves up for a whole new set of possible problems we are unfamiliar with. Whether that is an acceptable risk or not is a question for a different avenue however.
On Sun, Apr 3, 2022 at 7:10 PM Dmitriy Rabotyagov noonedeadpunk@gmail.com wrote:
Hey there!
I have quick question - do you think it's valid approach to install Ansible roles as python packages? This smells sooooo fishy since ansible-galaxy is a thing along with requirements.yml...
So actual question is - do you have any plans on changing this approach to more Ansible way anytime soon?
пт, 1 апр. 2022 г., 8:19 Marios Andreou marios@redhat.com:
On Fri, Apr 1, 2022 at 12:14 AM Ronelle Landy rlandy@redhat.com wrote:
Hello All,
We have a check/gate blocker on all TripleO quickstart-based jobs, as described in:
https://bugs.launchpad.net/tripleo/+bug/1967430
[1] commit to openstack-ansible-os_tempest removed setup.py and is causing failings in all quickstart jobs.
A revert was proposed but will not be workable - we are waiting on another fix.
Please hold rechecks until this is resolved.
[1] https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/835969
Unfortunately looks like the core group on that repo is empty [1]. I added some folks into CC here that merged the original patch. Folks can you please help us merge the fix at https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/8360...
TripleO gate is blocked until we merge ansible-role-python_venv_build/+/836091
please help :D
[1] https://review.opendev.org/admin/groups/3474fc86368161e5288be01295041a089a10...
Thank you!
On Mon, Apr 4, 2022 at 9:27 AM Jiri Podivin jpodivin@redhat.com wrote:
Full disclosure: I have only surface level understanding of how ansible galaxy actually works on the inside. My exposure to it is rather limited and it's possible that all of my concerns have perfectly valid responses I'm not aware of. Furthermore, I do believe that we could utilize ansible galaxy a bit more than we do.
That being said, I do think that we should be cautious when changing the way we package and deliver. Even if everything works out we are possibly setting ourselves up for a whole new set of possible problems we are unfamiliar with. Whether that is an acceptable risk or not is a question for a different avenue however.
In this particular case, we can get away with installing the ansible galaxy collections because we have 'nested' ansible so something like zuul (ansible) calling bash (tripleo-quickstart) calling ansible. There are other cases (zuul/ansible 'native', not nested) where we have to install such dependencies as python utilities because of the security concerns around allowing collections to be installed on the ansible controller (e.g. see http://lists.zuul-ci.org/pipermail/zuul-discuss/2021-November/001752.html).
In this case, we can do the installation of the required ansible bits during the middle "bash" part of the workflow (as you can see in https://review.opendev.org/c/openstack/tripleo-quickstart/+/836104). There are other cases where we can't (yet?)
regards, marios
On Sun, Apr 3, 2022 at 7:10 PM Dmitriy Rabotyagov noonedeadpunk@gmail.com wrote:
Hey there!
I have quick question - do you think it's valid approach to install Ansible roles as python packages? This smells sooooo fishy since ansible-galaxy is a thing along with requirements.yml...
So actual question is - do you have any plans on changing this approach to more Ansible way anytime soon?
пт, 1 апр. 2022 г., 8:19 Marios Andreou marios@redhat.com:
On Fri, Apr 1, 2022 at 12:14 AM Ronelle Landy rlandy@redhat.com wrote:
Hello All,
We have a check/gate blocker on all TripleO quickstart-based jobs, as described in:
https://bugs.launchpad.net/tripleo/+bug/1967430
[1] commit to openstack-ansible-os_tempest removed setup.py and is causing failings in all quickstart jobs.
A revert was proposed but will not be workable - we are waiting on another fix.
Please hold rechecks until this is resolved.
[1] https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/835969
Unfortunately looks like the core group on that repo is empty [1]. I added some folks into CC here that merged the original patch. Folks can you please help us merge the fix at https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/8360...
TripleO gate is blocked until we merge ansible-role-python_venv_build/+/836091
please help :D
[1] https://review.opendev.org/admin/groups/3474fc86368161e5288be01295041a089a10...
Thank you!
On 2022-04-04 09:35:39 +0300 (+0300), Marios Andreou wrote: [...]
In this particular case, we can get away with installing the ansible galaxy collections because we have 'nested' ansible so something like zuul (ansible) calling bash (tripleo-quickstart) calling ansible. There are other cases (zuul/ansible 'native', not nested) where we have to install such dependencies as python utilities because of the security concerns around allowing collections to be installed on the ansible controller (e.g. see http://lists.zuul-ci.org/pipermail/zuul-discuss/2021-November/001752.html).
[...]
We hope this will get simpler soon as we work toward Zuul v6:
https://zuul-ci.org/docs/zuul/latest/developer/specs/unrestricted-ansible.ht...
I understand. The question is how far back, if at all, should we backport the change. Provided that it is merged into master of course.
On Mon, Apr 4, 2022 at 2:21 PM Jeremy Stanley fungi@yuggoth.org wrote:
On 2022-04-04 09:35:39 +0300 (+0300), Marios Andreou wrote: [...]
In this particular case, we can get away with installing the ansible galaxy collections because we have 'nested' ansible so something like zuul (ansible) calling bash (tripleo-quickstart) calling ansible. There are other cases (zuul/ansible 'native', not nested) where we have to install such dependencies as python utilities because of the security concerns around allowing collections to be installed on the ansible controller (e.g. see
http://lists.zuul-ci.org/pipermail/zuul-discuss/2021-November/001752.html ). [...]
We hope this will get simpler soon as we work toward Zuul v6:
https://zuul-ci.org/docs/zuul/latest/developer/specs/unrestricted-ansible.ht...
-- Jeremy Stanley
On Mon, Apr 4, 2022 at 3:36 PM Jiri Podivin jpodivin@redhat.com wrote:
I understand. The question is how far back, if at all, should we backport the change. Provided that it is merged into master of course.
well the proposed fix on our side is in the ci tooling https://review.opendev.org/c/openstack/tripleo-quickstart/+/836104 and that repo is branchless so we should be good
On Mon, Apr 4, 2022 at 2:21 PM Jeremy Stanley fungi@yuggoth.org wrote:
On 2022-04-04 09:35:39 +0300 (+0300), Marios Andreou wrote: [...]
In this particular case, we can get away with installing the ansible galaxy collections because we have 'nested' ansible so something like zuul (ansible) calling bash (tripleo-quickstart) calling ansible. There are other cases (zuul/ansible 'native', not nested) where we have to install such dependencies as python utilities because of the security concerns around allowing collections to be installed on the ansible controller (e.g. see
http://lists.zuul-ci.org/pipermail/zuul-discuss/2021-November/001752.html ). [...]
We hope this will get simpler soon as we work toward Zuul v6:
https://zuul-ci.org/docs/zuul/latest/developer/specs/unrestricted-ansible.ht...
-- Jeremy Stanley
Right, makes sense.
On Mon, Apr 4, 2022 at 2:47 PM Marios Andreou marios@redhat.com wrote:
On Mon, Apr 4, 2022 at 3:36 PM Jiri Podivin jpodivin@redhat.com wrote:
I understand. The question is how far back, if at all, should we backport the change. Provided that it is merged into master of course.
well the proposed fix on our side is in the ci tooling https://review.opendev.org/c/openstack/tripleo-quickstart/+/836104 and that repo is branchless so we should be good
On Mon, Apr 4, 2022 at 2:21 PM Jeremy Stanley fungi@yuggoth.org wrote:
On 2022-04-04 09:35:39 +0300 (+0300), Marios Andreou wrote: [...]
In this particular case, we can get away with installing the ansible galaxy collections because we have 'nested' ansible so something like zuul (ansible) calling bash (tripleo-quickstart) calling ansible. There are other cases (zuul/ansible 'native', not nested) where we have to install such dependencies as python utilities because of the security concerns around allowing collections to be installed on the ansible controller (e.g. see
http://lists.zuul-ci.org/pipermail/zuul-discuss/2021-November/001752.html ). [...]
We hope this will get simpler soon as we work toward Zuul v6:
https://zuul-ci.org/docs/zuul/latest/developer/specs/unrestricted-ansible.ht...
-- Jeremy Stanley
On Mon, Apr 4, 2022 at 3:52 PM Jiri Podivin jpodivin@redhat.com wrote:
Right, makes sense.
On Mon, Apr 4, 2022 at 2:47 PM Marios Andreou marios@redhat.com wrote:
On Mon, Apr 4, 2022 at 3:36 PM Jiri Podivin jpodivin@redhat.com wrote:
I understand. The question is how far back, if at all, should we backport the change. Provided that it is merged into master of course.
well the proposed fix on our side is in the ci tooling https://review.opendev.org/c/openstack/tripleo-quickstart/+/836104 and that repo is branchless so we should be good
On Mon, Apr 4, 2022 at 2:21 PM Jeremy Stanley fungi@yuggoth.org wrote:
On 2022-04-04 09:35:39 +0300 (+0300), Marios Andreou wrote: [...]
In this particular case, we can get away with installing the ansible galaxy collections because we have 'nested' ansible so something like zuul (ansible) calling bash (tripleo-quickstart) calling ansible. There are other cases (zuul/ansible 'native', not nested) where we have to install such dependencies as python utilities because of the security concerns around allowing collections to be installed on the ansible controller (e.g. see
http://lists.zuul-ci.org/pipermail/zuul-discuss/2021-November/001752.html ). [...]
We hope this will get simpler soon as we work toward Zuul v6:
https://zuul-ci.org/docs/zuul/latest/developer/specs/unrestricted-ansible.ht...
forgot to add thanks for the pointer fungi - interesting - from a quick skim it doesn't appear to be completely unrestricted but will allow you to add some files/roles/collections into a special ("bubblewrap") env ? adding to reading list for more careful scanning later ;)
regards, marios
--
Jeremy Stanley
On 2022-04-04 15:58:19 +0300 (+0300), Marios Andreou wrote: [...]
from a quick skim it doesn't appear to be completely unrestricted but will allow you to add some files/roles/collections into a special ("bubblewrap") env ? adding to reading list for more careful scanning later ;)
Currently, the Zuul executors run Ansible in per-build containers in order to provide some separation so that jobs hopefully won't interfere with one another. In addition, Zuul uses a forked copy of Ansible's stdlib in order to prevent "unsafe" modules from being called in that container, or to remove "unsafe" features from some allowed modules.
What the spec proposes, in summary, is to drop that separate fork we're maintaining of the Ansible stdlib, and just allow jobs to call any module within the existing container on the executor.
The gate blocker is now cleared.
Thank you to all who got the required patches through.
On Fri, Apr 1, 2022 at 2:19 AM Marios Andreou marios@redhat.com wrote:
On Fri, Apr 1, 2022 at 12:14 AM Ronelle Landy rlandy@redhat.com wrote:
Hello All,
We have a check/gate blocker on all TripleO quickstart-based jobs, as described in:
https://bugs.launchpad.net/tripleo/+bug/1967430
[1] commit to openstack-ansible-os_tempest removed setup.py and is causing failings in all quickstart jobs.
A revert was proposed but will not be workable - we are waiting on another fix.
Please hold rechecks until this is resolved.
[1] https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/835969
Unfortunately looks like the core group on that repo is empty [1]. I added some folks into CC here that merged the original patch. Folks can you please help us merge the fix at https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/8360...
TripleO gate is blocked until we merge ansible-role-python_venv_build/+/836091
please help :D
[1] https://review.opendev.org/admin/groups/3474fc86368161e5288be01295041a089a10...
Thank you!
participants (5)
-
Dmitriy Rabotyagov
-
Jeremy Stanley
-
Jiri Podivin
-
Marios Andreou
-
Ronelle Landy