[tripleo] Package dependencies, podman, dnf modules
Hello there, Since we introduced Podman instead of Docker in tripleo, we are running after the right version. For now, the version we should use is provided by a specific container-tools module stream (container-tools:2.0). The default stream being container-tools:rhel8, we might end with a deployed undercloud running the wrong podman version, leading to some nice issues, especially in a SELinux-enabled environment. Currently, the main dependency tree is: python3-tripleoclient requires podman That's mostly all. While we can, of course, edit the python3-tripleoclient spec file in order to pin the precise podman version (or, at least, set upper and lower constraints) in order to target the version provided by the right stream, it's not ideal: - package install error will lead to more questions (since ppl might skip doc for #reason) - if we change the target version, we'll need to ensure it's present in a stream (and, well, update the doc if we need to switch the stream) - and probably some other reasons. Since we can NOT depend on a specific stream in rpm (for instance, we can't put "Requires @container-tools:2.0/podman" like we might in ansible), we're a bit stuck. An ugly hack is tested in the spec file[1], using a bash thing in order to check the activated stream, but this is not ideal either, especially since it makes the whole package building/checking fail during the "mock" stage. In order to make it pass, we'd need to modify the whole RDO in order to take into account stream switching. This isn't impossible, but it leads to other concerns, especially when it comes to "hey we need to switch the stream for <module>". And I'm pretty sure no one will want to dig into it, for good reasons ;). This leads to a last possibility: drop the "Requires podman" from tripleo dependency tree, and rely on tripleo-ansible to actually enable the correct module, and install podman. This means podman will be installed during the undercloud deploy, for instance as a host_prep_tasks (step_0). While I'm not that happy with this idea, it's probably not as bad as the other hacks we've tested so far, and would, at last, prevent once for all the usual "it's not working" due to a wrong podman version (and, yes, we get plenty of them, especially in the OSP version). What do you think? Any other way of solving this issue? Remember, we're talking about "ensuring we get the right version, coming from a specific stream" and, especially, "ensure operator can't install the wrong one" (if they don't follow the doc, of if they are using automation that doesn't check actual requirements in the doc"... Thank you for your thoughts! Cheers, C. [1] https://review.rdoproject.org/r/#/c/31411/ -- Cédric Jeanneret (He/Him/His) Sr. Software Engineer - OpenStack Platform Deployment Framework TC Red Hat EMEA https://www.redhat.com/
On Tue, Jan 5, 2021 at 9:44 AM Cédric Jeanneret <cjeanner@redhat.com> wrote:
Hello there,
Since we introduced Podman instead of Docker in tripleo, we are running after the right version. For now, the version we should use is provided by a specific container-tools module stream (container-tools:2.0). The default stream being container-tools:rhel8, we might end with a deployed undercloud running the wrong podman version, leading to some nice issues, especially in a SELinux-enabled environment.
o/ Tengu thanks for socialising that issue, it's a fun one for sure... first question is _where_ is that the default and it is mostly rhetorical as I can guess. In general though i think it's worth filing a LP bug with some of these details so we can also track whatever is decided as a fix here.
Currently, the main dependency tree is: python3-tripleoclient requires podman That's mostly all.
While we can, of course, edit the python3-tripleoclient spec file in order to pin the precise podman version (or, at least, set upper and lower constraints) in order to target the version provided by the right stream, it's not ideal: - package install error will lead to more questions (since ppl might skip doc for #reason) - if we change the target version, we'll need to ensure it's present in a stream (and, well, update the doc if we need to switch the stream) - and probably some other reasons.
Since we can NOT depend on a specific stream in rpm (for instance, we can't put "Requires @container-tools:2.0/podman" like we might in ansible), we're a bit stuck.
An ugly hack is tested in the spec file[1], using a bash thing in order to check the activated stream, but this is not ideal either, especially since it makes the whole package building/checking fail during the "mock" stage. In order to make it pass, we'd need to modify the whole RDO in order to take into account stream switching. This isn't impossible, but it leads to other concerns, especially when it comes to "hey we need to switch the stream for <module>". And I'm pretty sure no one will want to dig into it, for good reasons ;).
This leads to a last possibility: drop the "Requires podman" from tripleo dependency tree, and rely on
so how does that work in practice though I mean what about python-tripleoclient. Quick grep just now tells me at least the heat-launcher is directly using podman. How do we drop that requirement.
tripleo-ansible to actually enable the correct module, and install podman. This means podman will be installed during the undercloud deploy, for instance as a host_prep_tasks (step_0).
I am missing how using tripleo-ansible helps us with the stream question. Do you mean add logic in tripleo-ansible that tests the stream and sets the correct version ? While I'm not that happy with this idea, it's probably not as bad as the
other hacks we've tested so far, and would, at last, prevent once for all the usual "it's not working" due to a wrong podman version (and, yes, we get plenty of them, especially in the OSP version).
What do you think? Any other way of solving this issue? Remember, we're talking about "ensuring we get the right version, coming from a specific stream" and, especially, "ensure operator can't install the wrong one" (if they don't follow the doc, of if they are using automation that doesn't check actual requirements in the doc"...
in general i lean more towards 'ansible tasks' rather than 'bash thing in the specfile' to solve that. Either way though this is us reacting (with some fix/hack/workaround) to a problem that isn't anything to do with TripleO or even OpenStack but rather an OS/packaging issue. So do we have any longer term plans on what you mentioned on IRC just now regarding getting module support for RPM deps (do you mean something in the spec file here?) or is that a complete dead end? thanks, marios
Thank you for your thoughts!
Cheers,
C.
[1] https://review.rdoproject.org/r/#/c/31411/
-- Cédric Jeanneret (He/Him/His) Sr. Software Engineer - OpenStack Platform Deployment Framework TC Red Hat EMEA https://www.redhat.com/
On 1/5/21 9:56 AM, Marios Andreou wrote:
On Tue, Jan 5, 2021 at 9:44 AM Cédric Jeanneret <cjeanner@redhat.com <mailto:cjeanner@redhat.com>> wrote:
Hello there,
Since we introduced Podman instead of Docker in tripleo, we are running after the right version. For now, the version we should use is provided by a specific container-tools module stream (container-tools:2.0). The default stream being container-tools:rhel8, we might end with a deployed undercloud running the wrong podman version, leading to some nice issues, especially in a SELinux-enabled environment.
o/ Tengu thanks for socialising that issue, it's a fun one for sure...
first question is _where_ is that the default and it is mostly rhetorical as I can guess. In general though i think it's worth filing a LP bug with some of these details so we can also track whatever is decided as a fix here.
Right - always hard to find the right starting point ;). Though a mail would be a nice thing in order to get directions/ideas - and, hey, you just pointed some already! Here's the LP: https://bugs.launchpad.net/tripleo/+bug/1910217
Currently, the main dependency tree is: python3-tripleoclient requires podman That's mostly all.
While we can, of course, edit the python3-tripleoclient spec file in order to pin the precise podman version (or, at least, set upper and lower constraints) in order to target the version provided by the right stream, it's not ideal: - package install error will lead to more questions (since ppl might skip doc for #reason) - if we change the target version, we'll need to ensure it's present in a stream (and, well, update the doc if we need to switch the stream) - and probably some other reasons.
Since we can NOT depend on a specific stream in rpm (for instance, we can't put "Requires @container-tools:2.0/podman" like we might in ansible), we're a bit stuck.
An ugly hack is tested in the spec file[1], using a bash thing in order to check the activated stream, but this is not ideal either, especially since it makes the whole package building/checking fail during the "mock" stage. In order to make it pass, we'd need to modify the whole RDO in order to take into account stream switching. This isn't impossible, but it leads to other concerns, especially when it comes to "hey we need to switch the stream for <module>". And I'm pretty sure no one will want to dig into it, for good reasons ;).
This leads to a last possibility: drop the "Requires podman" from tripleo dependency tree, and rely on
so how does that work in practice though I mean what about python-tripleoclient. Quick grep just now tells me at least the heat-launcher is directly using podman. How do we drop that requirement.
Good catch. Maybe by ordering a bit things and calling the podman install bit before calling the heat-launcher? Might be ugly and, well, against some of the common practices, but at least it would ensure we get what we actually want regarding versions and sources.
tripleo-ansible to actually enable the correct module, and install podman. This means podman will be installed during the undercloud deploy, for instance as a host_prep_tasks (step_0).
I am missing how using tripleo-ansible helps us with the stream question. Do you mean add logic in tripleo-ansible that tests the stream and sets the correct version ?
exactly. Ansible is able to switch module streams, and install packages. Basically, we'd "just" need to extend a bit this role/task: https://opendev.org/openstack/tripleo-ansible/src/branch/master/tripleo_ansi... That would make things cleaner, nicer, and more reliable.
While I'm not that happy with this idea, it's probably not as bad as the other hacks we've tested so far, and would, at last, prevent once for all the usual "it's not working" due to a wrong podman version (and, yes, we get plenty of them, especially in the OSP version).
What do you think? Any other way of solving this issue? Remember, we're talking about "ensuring we get the right version, coming from a specific stream" and, especially, "ensure operator can't install the wrong one" (if they don't follow the doc, of if they are using automation that doesn't check actual requirements in the doc"...
in general i lean more towards 'ansible tasks' rather than 'bash thing in the specfile' to solve that. Either way though this is us reacting (with some fix/hack/workaround) to a problem that isn't anything to do with TripleO or even OpenStack but rather an OS/packaging issue. So do we have any longer term plans on what you mentioned on IRC just now regarding getting module support for RPM deps (do you mean something in the spec file here?) or is that a complete dead end?
AFAIK, module support within RPM spec file as a standard thing is not in the roadmap. Apparently, the ppl behind this "module" concept aren't willing to add any sort of support. I've worked with Lon (Red Hat side) for the patch in the spec file, we've tried multiple ways and, well...... it's really not something I'm happy with. One idea I just got was about rpm hooks - iirc there's a thing in dnf that might help, have to read some more things about that. But I have the feeling this is still the wrong way - unless it's considered as a "last security net just to be really-really sure", instead of the current usage we have (which is, basically, ensure ppl do read the doc and follow it). So the main thing: it's not a TripleO issue per se - we're just using a feature that has a weak support in some cases, and, well, we happen to hit one of those cases. As usuall I'd say ;). Cheers, C.
thanks, marios
Thank you for your thoughts!
Cheers,
C.
[1] https://review.rdoproject.org/r/#/c/31411/ <https://review.rdoproject.org/r/#/c/31411/>
-- Cédric Jeanneret (He/Him/His) Sr. Software Engineer - OpenStack Platform Deployment Framework TC Red Hat EMEA https://www.redhat.com/ <https://www.redhat.com/>
-- Cédric Jeanneret (He/Him/His) Sr. Software Engineer - OpenStack Platform Deployment Framework TC Red Hat EMEA https://www.redhat.com/
On 1/5/21 11:48 AM, Cédric Jeanneret wrote:
On 1/5/21 9:56 AM, Marios Andreou wrote:
On Tue, Jan 5, 2021 at 9:44 AM Cédric Jeanneret <cjeanner@redhat.com <mailto:cjeanner@redhat.com>> wrote:
Hello there,
Since we introduced Podman instead of Docker in tripleo, we are running after the right version. For now, the version we should use is provided by a specific container-tools module stream (container-tools:2.0). The default stream being container-tools:rhel8, we might end with a deployed undercloud running the wrong podman version, leading to some nice issues, especially in a SELinux-enabled environment.
o/ Tengu thanks for socialising that issue, it's a fun one for sure...
first question is _where_ is that the default and it is mostly rhetorical as I can guess. In general though i think it's worth filing a LP bug with some of these details so we can also track whatever is decided as a fix here.
Right - always hard to find the right starting point ;). Though a mail would be a nice thing in order to get directions/ideas - and, hey, you just pointed some already! Here's the LP: https://bugs.launchpad.net/tripleo/+bug/1910217
Currently, the main dependency tree is: python3-tripleoclient requires podman That's mostly all.
While we can, of course, edit the python3-tripleoclient spec file in order to pin the precise podman version (or, at least, set upper and lower constraints) in order to target the version provided by the right stream, it's not ideal: - package install error will lead to more questions (since ppl might skip doc for #reason) - if we change the target version, we'll need to ensure it's present in a stream (and, well, update the doc if we need to switch the stream) - and probably some other reasons.
Since we can NOT depend on a specific stream in rpm (for instance, we can't put "Requires @container-tools:2.0/podman" like we might in ansible), we're a bit stuck.
An ugly hack is tested in the spec file[1], using a bash thing in order to check the activated stream, but this is not ideal either, especially since it makes the whole package building/checking fail during the "mock" stage. In order to make it pass, we'd need to modify the whole RDO in order to take into account stream switching. This isn't impossible, but it leads to other concerns, especially when it comes to "hey we need to switch the stream for <module>". And I'm pretty sure no one will want to dig into it, for good reasons ;).
This leads to a last possibility: drop the "Requires podman" from tripleo dependency tree, and rely on
so how does that work in practice though I mean what about python-tripleoclient. Quick grep just now tells me at least the heat-launcher is directly using podman. How do we drop that requirement.
Good catch. Maybe by ordering a bit things and calling the podman install bit before calling the heat-launcher? Might be ugly and, well, against some of the common practices, but at least it would ensure we get what we actually want regarding versions and sources.
tripleo-ansible to actually enable the correct module, and install podman. This means podman will be installed during the undercloud deploy, for instance as a host_prep_tasks (step_0).
I am missing how using tripleo-ansible helps us with the stream question. Do you mean add logic in tripleo-ansible that tests the stream and sets the correct version ?
exactly. Ansible is able to switch module streams, and install packages. Basically, we'd "just" need to extend a bit this role/task: https://opendev.org/openstack/tripleo-ansible/src/branch/master/tripleo_ansi...
That would make things cleaner, nicer, and more reliable.
Maybe we can keep the dependency as-is, and "just" ensure tripleo-ansible "tripleo_podman" role sets the right things during the deploy time? For instance, we can add a task ensuring we're using the right stream, then install podman (and buildah). That means heat-launcher might use the wrong podman version (bleh...) but the finished undercloud should be fine? Not 100% sure it's really possible, especially since module switching might fail under some conditions when a package is installed :/. Guess some testing will be needed as well.
While I'm not that happy with this idea, it's probably not as bad as the other hacks we've tested so far, and would, at last, prevent once for all the usual "it's not working" due to a wrong podman version (and, yes, we get plenty of them, especially in the OSP version).
What do you think? Any other way of solving this issue? Remember, we're talking about "ensuring we get the right version, coming from a specific stream" and, especially, "ensure operator can't install the wrong one" (if they don't follow the doc, of if they are using automation that doesn't check actual requirements in the doc"...
in general i lean more towards 'ansible tasks' rather than 'bash thing in the specfile' to solve that. Either way though this is us reacting (with some fix/hack/workaround) to a problem that isn't anything to do with TripleO or even OpenStack but rather an OS/packaging issue. So do we have any longer term plans on what you mentioned on IRC just now regarding getting module support for RPM deps (do you mean something in the spec file here?) or is that a complete dead end?
AFAIK, module support within RPM spec file as a standard thing is not in the roadmap. Apparently, the ppl behind this "module" concept aren't willing to add any sort of support. I've worked with Lon (Red Hat side) for the patch in the spec file, we've tried multiple ways and, well...... it's really not something I'm happy with. One idea I just got was about rpm hooks - iirc there's a thing in dnf that might help, have to read some more things about that. But I have the feeling this is still the wrong way - unless it's considered as a "last security net just to be really-really sure", instead of the current usage we have (which is, basically, ensure ppl do read the doc and follow it).
So the main thing: it's not a TripleO issue per se - we're just using a feature that has a weak support in some cases, and, well, we happen to hit one of those cases. As usuall I'd say ;).
Cheers,
C.
thanks, marios
Thank you for your thoughts!
Cheers,
C.
[1] https://review.rdoproject.org/r/#/c/31411/ <https://review.rdoproject.org/r/#/c/31411/>
-- Cédric Jeanneret (He/Him/His) Sr. Software Engineer - OpenStack Platform Deployment Framework TC Red Hat EMEA https://www.redhat.com/ <https://www.redhat.com/>
-- Cédric Jeanneret (He/Him/His) Sr. Software Engineer - OpenStack Platform Deployment Framework TC Red Hat EMEA https://www.redhat.com/
On 1/5/21 12:02 PM, Cédric Jeanneret wrote:
On 1/5/21 11:48 AM, Cédric Jeanneret wrote:
On 1/5/21 9:56 AM, Marios Andreou wrote:
On Tue, Jan 5, 2021 at 9:44 AM Cédric Jeanneret <cjeanner@redhat.com <mailto:cjeanner@redhat.com>> wrote:
Hello there,
Since we introduced Podman instead of Docker in tripleo, we are running after the right version. For now, the version we should use is provided by a specific container-tools module stream (container-tools:2.0). The default stream being container-tools:rhel8, we might end with a deployed undercloud running the wrong podman version, leading to some nice issues, especially in a SELinux-enabled environment.
o/ Tengu thanks for socialising that issue, it's a fun one for sure...
first question is _where_ is that the default and it is mostly rhetorical as I can guess. In general though i think it's worth filing a LP bug with some of these details so we can also track whatever is decided as a fix here.
Right - always hard to find the right starting point ;). Though a mail would be a nice thing in order to get directions/ideas - and, hey, you just pointed some already! Here's the LP: https://bugs.launchpad.net/tripleo/+bug/1910217
Currently, the main dependency tree is: python3-tripleoclient requires podman That's mostly all.
While we can, of course, edit the python3-tripleoclient spec file in order to pin the precise podman version (or, at least, set upper and lower constraints) in order to target the version provided by the right stream, it's not ideal: - package install error will lead to more questions (since ppl might skip doc for #reason) - if we change the target version, we'll need to ensure it's present in a stream (and, well, update the doc if we need to switch the stream) - and probably some other reasons.
Since we can NOT depend on a specific stream in rpm (for instance, we can't put "Requires @container-tools:2.0/podman" like we might in ansible), we're a bit stuck.
An ugly hack is tested in the spec file[1], using a bash thing in order to check the activated stream, but this is not ideal either, especially since it makes the whole package building/checking fail during the "mock" stage. In order to make it pass, we'd need to modify the whole RDO in order to take into account stream switching. This isn't impossible, but it leads to other concerns, especially when it comes to "hey we need to switch the stream for <module>". And I'm pretty sure no one will want to dig into it, for good reasons ;).
This leads to a last possibility: drop the "Requires podman" from tripleo dependency tree, and rely on
so how does that work in practice though I mean what about python-tripleoclient. Quick grep just now tells me at least the heat-launcher is directly using podman. How do we drop that requirement.
Good catch. Maybe by ordering a bit things and calling the podman install bit before calling the heat-launcher? Might be ugly and, well, against some of the common practices, but at least it would ensure we get what we actually want regarding versions and sources.
tripleo-ansible to actually enable the correct module, and install podman. This means podman will be installed during the undercloud deploy, for instance as a host_prep_tasks (step_0).
I am missing how using tripleo-ansible helps us with the stream question. Do you mean add logic in tripleo-ansible that tests the stream and sets the correct version ?
exactly. Ansible is able to switch module streams, and install packages. Basically, we'd "just" need to extend a bit this role/task: https://opendev.org/openstack/tripleo-ansible/src/branch/master/tripleo_ansi...
That would make things cleaner, nicer, and more reliable.
Maybe we can keep the dependency as-is, and "just" ensure tripleo-ansible "tripleo_podman" role sets the right things during the deploy time? For instance, we can add a task ensuring we're using the right stream, then install podman (and buildah). That means heat-launcher might use the wrong podman version (bleh...) but the finished undercloud should be fine? Not 100% sure it's really possible, especially since module switching might fail under some conditions when a package is installed :/.
I propose to use in RPM specs Requires: podman Conflicts: podman < some_lower_constraint_version that would allow to fail early if, a wrong version is going to be installed via existing repos and streams. And failing early is much better than waiting up to the moment it starts running ansible IMHO. And at that later point it would only re-ensure the wanted version.
Guess some testing will be needed as well.
While I'm not that happy with this idea, it's probably not as bad as the other hacks we've tested so far, and would, at last, prevent once for all the usual "it's not working" due to a wrong podman version (and, yes, we get plenty of them, especially in the OSP version).
What do you think? Any other way of solving this issue? Remember, we're talking about "ensuring we get the right version, coming from a specific stream" and, especially, "ensure operator can't install the wrong one" (if they don't follow the doc, of if they are using automation that doesn't check actual requirements in the doc"...
in general i lean more towards 'ansible tasks' rather than 'bash thing in the specfile' to solve that. Either way though this is us reacting (with some fix/hack/workaround) to a problem that isn't anything to do with TripleO or even OpenStack but rather an OS/packaging issue. So do we have any longer term plans on what you mentioned on IRC just now regarding getting module support for RPM deps (do you mean something in the spec file here?) or is that a complete dead end?
AFAIK, module support within RPM spec file as a standard thing is not in the roadmap. Apparently, the ppl behind this "module" concept aren't willing to add any sort of support. I've worked with Lon (Red Hat side) for the patch in the spec file, we've tried multiple ways and, well...... it's really not something I'm happy with. One idea I just got was about rpm hooks - iirc there's a thing in dnf that might help, have to read some more things about that. But I have the feeling this is still the wrong way - unless it's considered as a "last security net just to be really-really sure", instead of the current usage we have (which is, basically, ensure ppl do read the doc and follow it).
So the main thing: it's not a TripleO issue per se - we're just using a feature that has a weak support in some cases, and, well, we happen to hit one of those cases. As usuall I'd say ;).
Cheers,
C.
thanks, marios
Thank you for your thoughts!
Cheers,
C.
[1] https://review.rdoproject.org/r/#/c/31411/ <https://review.rdoproject.org/r/#/c/31411/>
-- Cédric Jeanneret (He/Him/His) Sr. Software Engineer - OpenStack Platform Deployment Framework TC Red Hat EMEA https://www.redhat.com/ <https://www.redhat.com/>
-- Best regards, Bogdan Dobrelya, Irc #bogdando
On 1/5/21 12:45 PM, Bogdan Dobrelya wrote:
On 1/5/21 12:02 PM, Cédric Jeanneret wrote:
On 1/5/21 11:48 AM, Cédric Jeanneret wrote:
On 1/5/21 9:56 AM, Marios Andreou wrote:
On Tue, Jan 5, 2021 at 9:44 AM Cédric Jeanneret <cjeanner@redhat.com <mailto:cjeanner@redhat.com>> wrote:
Hello there,
Since we introduced Podman instead of Docker in tripleo, we are running after the right version. For now, the version we should use is provided by a specific container-tools module stream (container-tools:2.0). The default stream being container-tools:rhel8, we might end with a deployed undercloud running the wrong podman version, leading to some nice issues, especially in a SELinux-enabled environment.
o/ Tengu thanks for socialising that issue, it's a fun one for sure...
first question is _where_ is that the default and it is mostly rhetorical as I can guess. In general though i think it's worth filing a LP bug with some of these details so we can also track whatever is decided as a fix here.
Right - always hard to find the right starting point ;). Though a mail would be a nice thing in order to get directions/ideas - and, hey, you just pointed some already! Here's the LP: https://bugs.launchpad.net/tripleo/+bug/1910217
Currently, the main dependency tree is: python3-tripleoclient requires podman That's mostly all.
While we can, of course, edit the python3-tripleoclient spec file in order to pin the precise podman version (or, at least, set upper and lower constraints) in order to target the version provided by the right stream, it's not ideal: - package install error will lead to more questions (since ppl might skip doc for #reason) - if we change the target version, we'll need to ensure it's present in a stream (and, well, update the doc if we need to switch the stream) - and probably some other reasons.
Since we can NOT depend on a specific stream in rpm (for instance, we can't put "Requires @container-tools:2.0/podman" like we might in ansible), we're a bit stuck.
An ugly hack is tested in the spec file[1], using a bash thing in order to check the activated stream, but this is not ideal either, especially since it makes the whole package building/checking fail during the "mock" stage. In order to make it pass, we'd need to modify the whole RDO in order to take into account stream switching. This isn't impossible, but it leads to other concerns, especially when it comes to "hey we need to switch the stream for <module>". And I'm pretty sure no one will want to dig into it, for good reasons ;).
This leads to a last possibility: drop the "Requires podman" from tripleo dependency tree, and rely on
so how does that work in practice though I mean what about python-tripleoclient. Quick grep just now tells me at least the heat-launcher is directly using podman. How do we drop that requirement.
Good catch. Maybe by ordering a bit things and calling the podman install bit before calling the heat-launcher? Might be ugly and, well, against some of the common practices, but at least it would ensure we get what we actually want regarding versions and sources.
tripleo-ansible to actually enable the correct module, and install podman. This means podman will be installed during the undercloud deploy, for instance as a host_prep_tasks (step_0).
I am missing how using tripleo-ansible helps us with the stream question. Do you mean add logic in tripleo-ansible that tests the stream and sets the correct version ?
exactly. Ansible is able to switch module streams, and install packages. Basically, we'd "just" need to extend a bit this role/task: https://opendev.org/openstack/tripleo-ansible/src/branch/master/tripleo_ansi...
That would make things cleaner, nicer, and more reliable.
Maybe we can keep the dependency as-is, and "just" ensure tripleo-ansible "tripleo_podman" role sets the right things during the deploy time? For instance, we can add a task ensuring we're using the right stream, then install podman (and buildah). That means heat-launcher might use the wrong podman version (bleh...) but the finished undercloud should be fine? Not 100% sure it's really possible, especially since module switching might fail under some conditions when a package is installed :/.
I propose to use in RPM specs
Requires: podman Conflicts: podman < some_lower_constraint_version
that would allow to fail early if, a wrong version is going to be installed via existing repos and streams. And failing early is much better than waiting up to the moment it starts running ansible IMHO. And at that later point it would only re-ensure the wanted version.
We'll need to set an upper constraint - in some conditions, the default version is newer than the one we actually want. This is already done, more or less, downstream with an OSP specific package. While it sounds appealing, I still see issues when it comes to version changes. I'd rather ensure we have the proper stream activated, especially since your proposal will lead to more LP/BZ being opened by ppl that don't read the doc© - resulting in endless bounces and more complains about the non-user-friendliness of this approach :(. Ah, also.... we seem to have the same kind of constrain with some "virt" module - default is "rhel" but we apparently need another one (iirc "8") - though this seems to be downstream-only, and only for the overcloud (or standalone)... So the issue isn't "just" for podman, and a better, generic thing would be really nice :). Cheers, C.
Guess some testing will be needed as well.
While I'm not that happy with this idea, it's probably not as bad as the other hacks we've tested so far, and would, at last, prevent once for all the usual "it's not working" due to a wrong podman version (and, yes, we get plenty of them, especially in the OSP version).
What do you think? Any other way of solving this issue? Remember, we're talking about "ensuring we get the right version, coming from a specific stream" and, especially, "ensure operator can't install the wrong one" (if they don't follow the doc, of if they are using automation that doesn't check actual requirements in the doc"...
in general i lean more towards 'ansible tasks' rather than 'bash thing in the specfile' to solve that. Either way though this is us reacting (with some fix/hack/workaround) to a problem that isn't anything to do with TripleO or even OpenStack but rather an OS/packaging issue. So do we have any longer term plans on what you mentioned on IRC just now regarding getting module support for RPM deps (do you mean something in the spec file here?) or is that a complete dead end?
AFAIK, module support within RPM spec file as a standard thing is not in the roadmap. Apparently, the ppl behind this "module" concept aren't willing to add any sort of support. I've worked with Lon (Red Hat side) for the patch in the spec file, we've tried multiple ways and, well...... it's really not something I'm happy with. One idea I just got was about rpm hooks - iirc there's a thing in dnf that might help, have to read some more things about that. But I have the feeling this is still the wrong way - unless it's considered as a "last security net just to be really-really sure", instead of the current usage we have (which is, basically, ensure ppl do read the doc and follow it).
So the main thing: it's not a TripleO issue per se - we're just using a feature that has a weak support in some cases, and, well, we happen to hit one of those cases. As usuall I'd say ;).
Cheers,
C.
thanks, marios
Thank you for your thoughts!
Cheers,
C.
[1] https://review.rdoproject.org/r/#/c/31411/ <https://review.rdoproject.org/r/#/c/31411/>
-- Cédric Jeanneret (He/Him/His) Sr. Software Engineer - OpenStack Platform Deployment Framework TC Red Hat EMEA https://www.redhat.com/ <https://www.redhat.com/>
-- Cédric Jeanneret (He/Him/His) Sr. Software Engineer - OpenStack Platform Deployment Framework TC Red Hat EMEA https://www.redhat.com/
On Tue, Jan 5, 2021 at 12:53 AM Cédric Jeanneret <cjeanner@redhat.com> wrote:
Hello there,
Since we introduced Podman instead of Docker in tripleo, we are running after the right version. For now, the version we should use is provided by a specific container-tools module stream (container-tools:2.0). The default stream being container-tools:rhel8, we might end with a deployed undercloud running the wrong podman version, leading to some nice issues, especially in a SELinux-enabled environment.
Currently, the main dependency tree is: python3-tripleoclient requires podman That's mostly all.
While we can, of course, edit the python3-tripleoclient spec file in order to pin the precise podman version (or, at least, set upper and lower constraints) in order to target the version provided by the right stream, it's not ideal: - package install error will lead to more questions (since ppl might skip doc for #reason) - if we change the target version, we'll need to ensure it's present in a stream (and, well, update the doc if we need to switch the stream) - and probably some other reasons.
So as mentioned we did this and it leads to poor UX because the conflict message doesn't help direct people to fixing their container-tools module.
Since we can NOT depend on a specific stream in rpm (for instance, we can't put "Requires @container-tools:2.0/podman" like we might in ansible), we're a bit stuck.
IMHO This really needs to be in the RPM spec ASAP to make modules viable in the long term but that's not an OpenStack issue.
An ugly hack is tested in the spec file[1], using a bash thing in order to check the activated stream, but this is not ideal either, especially since it makes the whole package building/checking fail during the "mock" stage. In order to make it pass, we'd need to modify the whole RDO in order to take into account stream switching. This isn't impossible, but it leads to other concerns, especially when it comes to "hey we need to switch the stream for <module>". And I'm pretty sure no one will want to dig into it, for good reasons ;).
This leads to a last possibility: drop the "Requires podman" from tripleo dependency tree, and rely on tripleo-ansible to actually enable the correct module, and install podman. This means podman will be installed during the undercloud deploy, for instance as a host_prep_tasks (step_0).
Since we may or may not consume podman in code we're packaging we likely should not drop the requirement.
While I'm not that happy with this idea, it's probably not as bad as the other hacks we've tested so far, and would, at last, prevent once for all the usual "it's not working" due to a wrong podman version (and, yes, we get plenty of them, especially in the OSP version).
What do you think? Any other way of solving this issue? Remember, we're talking about "ensuring we get the right version, coming from a specific stream" and, especially, "ensure operator can't install the wrong one" (if they don't follow the doc, of if they are using automation that doesn't check actual requirements in the doc"...
IMHO, as mentioned on IRC, we need to improve the initial setup process to reduce the likelihood of a misconfiguration. I had chatted with the CI folks on the best way to solve this in a single place that would be useful for users and in CI. We generally agreed to combine the efforts into tripleo-repos because it can be installed independently of everything else and can be a single source of truth for version info. I proposed a WIP for a version information structure so we can have a single source of truth for repo and version information. https://review.opendev.org/c/openstack/tripleo-repos/+/767214 Today we require the user to manually setup repositories and module versions prior to installing the packages. I think it's best if we work on reducing the errors that occur during this process. We could add something into the validation bits, but we will still need a single source of truth for the required versions so we're not duplicating the version information. Thanks, -Alex
Thank you for your thoughts!
Cheers,
C.
[1] https://review.rdoproject.org/r/#/c/31411/
-- Cédric Jeanneret (He/Him/His) Sr. Software Engineer - OpenStack Platform Deployment Framework TC Red Hat EMEA https://www.redhat.com/
participants (4)
-
Alex Schultz
-
Bogdan Dobrelya
-
Cédric Jeanneret
-
Marios Andreou