[tripleo][ansible] the current action plugins use patterns are suboptimal?
There is a trend of writing action plugins, see [0], for simple things, like just calling a module in a loop. I'm not sure that is the direction TripleO should go. If ansible is inefficient in this sort of tasks without custom python code written, we should fix ansible. Otherwise, what is the ultimate goal of that trend? Is that having only action plugins in roles and playbooks? Please kindly asking the community to stop that, make a step back and reiterate with the taken approach. Thank you. [0] https://review.opendev.org/716108 -- Best regards, Bogdan Dobrelya, Irc #bogdando
Hi, Bogdan thanks for raising this up, although I'm not sure I understand what it is the problem with using action plugins. Action plugins are well known official extensions for Ansible, as any other plugins - callback, strategy, inventory etc [1]. It is not any hack or unsupported workaround, it's a known and official feature of Ansible. Why can't we use it? What makes it different from filter, lookup, inventory or any other plugin we already use? Action plugins are also used wide in Ansible itself, for example templates plugin is implemented with action plugin [2]. If Ansible can use it, why can't we? I don't think there is something with "fixing" Ansible, it's not a bug, this is a useful extension. What regards the mentioned action plugin for podman containers, it allows to spawn containers remotely while skipping the connection part for every cycle. I'm not sure you can "fix" Ansible not to do that, it's not a bug. We may not see the difference in a few hosts in CI, but it might be very efficient when we deploy on 100+ hosts oro even 1000+ hosts. In order to evaluate this on bigger setups to understand its value we configured both options - to use action plugin or usual module. If better performance of action plugin will be proven, we can switch to use it, if it doesn't make a difference on bigger setups - then I think we can easily switch back to using an usual module. Thanks [1] https://docs.ansible.com/ansible/latest/plugins/plugins.html [2] https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/action/tem... On Mon, Aug 3, 2020 at 11:19 AM Bogdan Dobrelya <bdobreli@redhat.com> wrote:
There is a trend of writing action plugins, see [0], for simple things, like just calling a module in a loop. I'm not sure that is the direction TripleO should go. If ansible is inefficient in this sort of tasks without custom python code written, we should fix ansible. Otherwise, what is the ultimate goal of that trend? Is that having only action plugins in roles and playbooks?
Please kindly asking the community to stop that, make a step back and reiterate with the taken approach. Thank you.
[0] https://review.opendev.org/716108
-- Best regards, Bogdan Dobrelya, Irc #bogdando
-- Best regards Sagi Shnaidman
On 8/3/20 12:36 PM, Sagi Shnaidman wrote:
Hi, Bogdan
thanks for raising this up, although I'm not sure I understand what it is the problem with using action plugins. Action plugins are well known official extensions for Ansible, as any other plugins - callback, strategy, inventory etc [1]. It is not any hack or unsupported workaround, it's a known and official feature of Ansible. Why can't we use it? What makes it different from filter,
I believe the cases that require the use of those should be justified. For the given example, that manages containers in a loop via calling a module, what the written custom callback plugin buys for us? That brings code to maintain, extra complexity, like handling possible corner cases in async mode, dry-run mode etc. But what is justification aside of looks handy?
lookup, inventory or any other plugin we already use? Action plugins are also used wide in Ansible itself, for example templates plugin is implemented with action plugin [2]. If Ansible can use it, why can't we? I don't think there is something with "fixing" Ansible, it's not a bug, this is a useful extension. What regards the mentioned action plugin for podman containers, it allows to spawn containers remotely while skipping the connection part for every cycle. I'm not sure you can "fix" Ansible not to do that, it's not a bug. We may not see the difference in a few hosts in CI, but it might be very efficient when we deploy on 100+ hosts oro even 1000+ hosts. In order to evaluate this on bigger setups to understand its value we configured both options - to use action plugin or usual module. If better performance of action plugin will be proven, we can switch to use it, if it doesn't make a difference on bigger setups - then I think we can easily switch back to using an usual module.
Thanks
[1] https://docs.ansible.com/ansible/latest/plugins/plugins.html [2] https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/action/tem...
On Mon, Aug 3, 2020 at 11:19 AM Bogdan Dobrelya <bdobreli@redhat.com <mailto:bdobreli@redhat.com>> wrote:
There is a trend of writing action plugins, see [0], for simple things, like just calling a module in a loop. I'm not sure that is the direction TripleO should go. If ansible is inefficient in this sort of tasks without custom python code written, we should fix ansible. Otherwise, what is the ultimate goal of that trend? Is that having only action plugins in roles and playbooks?
Please kindly asking the community to stop that, make a step back and reiterate with the taken approach. Thank you.
[0] https://review.opendev.org/716108
-- Best regards, Bogdan Dobrelya, Irc #bogdando
-- Best regards Sagi Shnaidman
-- Best regards, Bogdan Dobrelya, Irc #bogdando
On Mon, Aug 3, 2020 at 6:34 AM Bogdan Dobrelya <bdobreli@redhat.com> wrote:
On 8/3/20 12:36 PM, Sagi Shnaidman wrote:
Hi, Bogdan
thanks for raising this up, although I'm not sure I understand what it is the problem with using action plugins. Action plugins are well known official extensions for Ansible, as any other plugins - callback, strategy, inventory etc [1]. It is not any hack or unsupported workaround, it's a known and official feature of Ansible. Why can't we use it? What makes it different from filter,
I believe the cases that require the use of those should be justified. For the given example, that manages containers in a loop via calling a module, what the written custom callback plugin buys for us? That brings code to maintain, extra complexity, like handling possible corner cases in async mode, dry-run mode etc. But what is justification aside of looks handy?
I disagree that we shouldn't use action plugins or modules. Tasks themselves are expensive at scale. We saw that when we switched away from paunch to container management in pure ansible tasks. This exposed that looping tasks are even more expensive and complex error handling and workflows are better suited for modules or action plugins than a series of tasks. This is not something to be "fixed in ansible". This is the nature of the executor and strategy related interactions. Should everything be converted to modules and plugins? no. Should everything be tasks only? no. It's a balance that must be struck between when a specific set of complex tasks need extra data processing or error handling. Switching to modules or action plugins allows us to unit test our logic. Using tasks do not have such a concept outside of writing complex molecule testing. IMHO it's safer to switch to modules/action plugins than writing task logic. IMHO the issue that I see with the switch to Action plugins is the increased load on the ansible "controller" node during execution. Modules may be better depending on the task being managed. But I believe with unit testing, action plugins or modules provide a cleaner and more testable solution than writing roles consisting only of tasks.
lookup, inventory or any other plugin we already use? Action plugins are also used wide in Ansible itself, for example templates plugin is implemented with action plugin [2]. If Ansible can use it, why can't we? I don't think there is something with "fixing" Ansible, it's not a bug, this is a useful extension. What regards the mentioned action plugin for podman containers, it allows to spawn containers remotely while skipping the connection part for every cycle. I'm not sure you can "fix" Ansible not to do that, it's not a bug. We may not see the difference in a few hosts in CI, but it might be very efficient when we deploy on 100+ hosts oro even 1000+ hosts. In order to evaluate this on bigger setups to understand its value we configured both options - to use action plugin or usual module. If better performance of action plugin will be proven, we can switch to use it, if it doesn't make a difference on bigger setups - then I think we can easily switch back to using an usual module.
Thanks
[1] https://docs.ansible.com/ansible/latest/plugins/plugins.html [2] https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/action/tem...
On Mon, Aug 3, 2020 at 11:19 AM Bogdan Dobrelya <bdobreli@redhat.com <mailto:bdobreli@redhat.com>> wrote:
There is a trend of writing action plugins, see [0], for simple things, like just calling a module in a loop. I'm not sure that is the direction TripleO should go. If ansible is inefficient in this sort of tasks without custom python code written, we should fix ansible. Otherwise, what is the ultimate goal of that trend? Is that having only action plugins in roles and playbooks?
Please kindly asking the community to stop that, make a step back and reiterate with the taken approach. Thank you.
[0] https://review.opendev.org/716108
-- Best regards, Bogdan Dobrelya, Irc #bogdando
-- Best regards Sagi Shnaidman
-- Best regards, Bogdan Dobrelya, Irc #bogdando
On Mon, 2020-08-03 at 13:28 -0600, Alex Schultz wrote: On Mon, Aug 3, 2020 at 6:34 AM Bogdan Dobrelya < <mailto:bdobreli@redhat.com> bdobreli@redhat.com
wrote:
On 8/3/20 12:36 PM, Sagi Shnaidman wrote: Hi, Bogdan thanks for raising this up, although I'm not sure I understand what it is the problem with using action plugins. Action plugins are well known official extensions for Ansible, as any other plugins - callback, strategy, inventory etc [1]. It is not any hack or unsupported workaround, it's a known and official feature of Ansible. Why can't we use it? What makes it different from filter, I believe the cases that require the use of those should be justified. For the given example, that manages containers in a loop via calling a module, what the written custom callback plugin buys for us? That brings code to maintain, extra complexity, like handling possible corner cases in async mode, dry-run mode etc. But what is justification aside of looks handy? I disagree that we shouldn't use action plugins or modules. Tasks themselves are expensive at scale. We saw that when we switched away from paunch to container management in pure ansible tasks. This exposed that looping tasks are even more expensive and complex error handling and workflows are better suited for modules or action plugins than a series of tasks. This is not something to be "fixed in ansible". This is the nature of the executor and strategy related interactions. Should everything be converted to modules and plugins? no. Should everything be tasks only? no. It's a balance that must be struck between when a specific set of complex tasks need extra data processing or error handling. Switching to modules or action plugins allows us to unit test our logic. Using tasks do not have such a concept outside of writing complex molecule testing. IMHO it's safer to switch to modules/action plugins than writing task logic. I agree with Alex. Writing complex logic or trying to do error handling in tasks or jinja is not only very slow in execution, but gives us no way to properly test. Using ansible extensions like modules, action plugins, filters, etc gives us something that we can unit test, do better error handling with and therefore provides the abilty to produce a better quality result. While it is true that it does give us more of a downstream burden, our community is well versed in reading python code and testing python code properly. Sometimes it might seem easier to an author to prototype something using ansible/jinja, but if the result is complex then an extension of some kind should be considered as a iteration and it should be unit tested. I'd go as far as to say that if we add module, we should force the requirement for unit testing through some means to ensure good code quality and maintainability. Another benefit to using modules is that the Ansible tasks read more like a sequence of events that need to happen, which is exactly the spirit that Ansible has always advocated. When complex logic is implemented in tasks or in jinja, trying to follow the orchestration sequence becomes a *lot* harder.
On 8/3/20 9:28 PM, Alex Schultz wrote:
On Mon, Aug 3, 2020 at 6:34 AM Bogdan Dobrelya <bdobreli@redhat.com> wrote:
On 8/3/20 12:36 PM, Sagi Shnaidman wrote:
Hi, Bogdan
thanks for raising this up, although I'm not sure I understand what it is the problem with using action plugins. Action plugins are well known official extensions for Ansible, as any other plugins - callback, strategy, inventory etc [1]. It is not any hack or unsupported workaround, it's a known and official feature of Ansible. Why can't we use it? What makes it different from filter,
I believe the cases that require the use of those should be justified. For the given example, that manages containers in a loop via calling a module, what the written custom callback plugin buys for us? That brings code to maintain, extra complexity, like handling possible corner cases in async mode, dry-run mode etc. But what is justification aside of looks handy?
I disagree that we shouldn't use action plugins or modules. Tasks themselves are expensive at scale. We saw that when we switched away from paunch to container management in pure ansible tasks. This exposed that looping tasks are even more expensive and complex error handling and workflows are better suited for modules or action plugins than a series of tasks. This is not something to be "fixed in ansible". This is the nature of the executor and strategy related interactions. Should everything be converted to modules and plugins? no. Should everything be tasks only? no. It's a balance that must be struck between when a specific set of complex tasks need extra data processing or error handling. Switching to modules or action plugins allows us to unit test our logic. Using tasks do not have such a
I can understand that ansible should not be fixed for some composition tasks what require iterations and have complex logic for its "unit of work". And such ones also should be unit tested indeed. What I do not fully understand though is then what abandoning paunch for its action plugin had bought for us in the end? Paunch was self-contained and had no external dependencies on fast-changing ansible frameworks. There was also no need for paunch to handle the ansible-specific execution strategies and nuances, like "what if that action plugin is called in async or in the check-mode?" Unit tests exited in paunch as well. It was easy to backport changes within a single code base. So, looking back retrospectively, was rewriting paunch as an action plugin a simplification of the deployment framework? Please reply to yourself honestly. It does pretty same things but differently and added external framework. It is now also self-contained action plugin, since traditional tasks cannot be used in loops for this goal because of performance reasons. To summarize, action plugins may be a good solution indeed, but perhaps we should go back and use paunch instead of ansible? Same applies for *some* other tasks? That would also provide a balance, for action plugins, tasks and common sense.
concept outside of writing complex molecule testing. IMHO it's safer to switch to modules/action plugins than writing task logic.
IMHO the issue that I see with the switch to Action plugins is the increased load on the ansible "controller" node during execution. Modules may be better depending on the task being managed. But I believe with unit testing, action plugins or modules provide a cleaner and more testable solution than writing roles consisting only of tasks.
lookup, inventory or any other plugin we already use? Action plugins are also used wide in Ansible itself, for example templates plugin is implemented with action plugin [2]. If Ansible can use it, why can't we? I don't think there is something with "fixing" Ansible, it's not a bug, this is a useful extension. What regards the mentioned action plugin for podman containers, it allows to spawn containers remotely while skipping the connection part for every cycle. I'm not sure you can "fix" Ansible not to do that, it's not a bug. We may not see the difference in a few hosts in CI, but it might be very efficient when we deploy on 100+ hosts oro even 1000+ hosts. In order to evaluate this on bigger setups to understand its value we configured both options - to use action plugin or usual module. If better performance of action plugin will be proven, we can switch to use it, if it doesn't make a difference on bigger setups - then I think we can easily switch back to using an usual module.
Thanks
[1] https://docs.ansible.com/ansible/latest/plugins/plugins.html [2] https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/action/tem...
On Mon, Aug 3, 2020 at 11:19 AM Bogdan Dobrelya <bdobreli@redhat.com <mailto:bdobreli@redhat.com>> wrote:
There is a trend of writing action plugins, see [0], for simple things, like just calling a module in a loop. I'm not sure that is the direction TripleO should go. If ansible is inefficient in this sort of tasks without custom python code written, we should fix ansible. Otherwise, what is the ultimate goal of that trend? Is that having only action plugins in roles and playbooks?
Please kindly asking the community to stop that, make a step back and reiterate with the taken approach. Thank you.
[0] https://review.opendev.org/716108
-- Best regards, Bogdan Dobrelya, Irc #bogdando
-- Best regards Sagi Shnaidman
-- Best regards, Bogdan Dobrelya, Irc #bogdando
-- Best regards, Bogdan Dobrelya, Irc #bogdando
Hi, Actually this discussion prompted me to investigate more how to optimize containers setup on a large number of hosts. As I saw from action plugin work, it still copies the module file each loop cycle, which is not very efficient behavior. That's why I started work on a podman container module which can start a bunch of containers in one call and accepts a list of containers as an input. In this case the module file will be transferred to the remote host only once and all containers execution will be done by python on the remote host. That way we'll avoid unnecessary establishing connections, copying files, setting permissions etc what happens every time we cycle over data. It will be done only once. I'll send a patch soon for review. Thanks On Tue, Aug 4, 2020 at 12:35 PM Bogdan Dobrelya <bdobreli@redhat.com> wrote:
On 8/3/20 9:28 PM, Alex Schultz wrote:
On Mon, Aug 3, 2020 at 6:34 AM Bogdan Dobrelya <bdobreli@redhat.com> wrote:
On 8/3/20 12:36 PM, Sagi Shnaidman wrote:
Hi, Bogdan
thanks for raising this up, although I'm not sure I understand what it is the problem with using action plugins. Action plugins are well known official extensions for Ansible, as any other plugins - callback, strategy, inventory etc [1]. It is not any hack or unsupported workaround, it's a known and official feature of Ansible. Why can't we use it? What makes it different from filter,
I believe the cases that require the use of those should be justified. For the given example, that manages containers in a loop via calling a module, what the written custom callback plugin buys for us? That brings code to maintain, extra complexity, like handling possible corner cases in async mode, dry-run mode etc. But what is justification aside of looks handy?
I disagree that we shouldn't use action plugins or modules. Tasks themselves are expensive at scale. We saw that when we switched away from paunch to container management in pure ansible tasks. This exposed that looping tasks are even more expensive and complex error handling and workflows are better suited for modules or action plugins than a series of tasks. This is not something to be "fixed in ansible". This is the nature of the executor and strategy related interactions. Should everything be converted to modules and plugins? no. Should everything be tasks only? no. It's a balance that must be struck between when a specific set of complex tasks need extra data processing or error handling. Switching to modules or action plugins allows us to unit test our logic. Using tasks do not have such a
I can understand that ansible should not be fixed for some composition tasks what require iterations and have complex logic for its "unit of work". And such ones also should be unit tested indeed. What I do not fully understand though is then what abandoning paunch for its action plugin had bought for us in the end?
Paunch was self-contained and had no external dependencies on fast-changing ansible frameworks. There was also no need for paunch to handle the ansible-specific execution strategies and nuances, like "what if that action plugin is called in async or in the check-mode?" Unit tests exited in paunch as well. It was easy to backport changes within a single code base.
So, looking back retrospectively, was rewriting paunch as an action plugin a simplification of the deployment framework? Please reply to yourself honestly. It does pretty same things but differently and added external framework. It is now also self-contained action plugin, since traditional tasks cannot be used in loops for this goal because of performance reasons.
To summarize, action plugins may be a good solution indeed, but perhaps we should go back and use paunch instead of ansible? Same applies for *some* other tasks? That would also provide a balance, for action plugins, tasks and common sense.
concept outside of writing complex molecule testing. IMHO it's safer to switch to modules/action plugins than writing task logic.
IMHO the issue that I see with the switch to Action plugins is the increased load on the ansible "controller" node during execution. Modules may be better depending on the task being managed. But I believe with unit testing, action plugins or modules provide a cleaner and more testable solution than writing roles consisting only of tasks.
lookup, inventory or any other plugin we already use? Action plugins are also used wide in Ansible itself, for example templates plugin is implemented with action plugin [2]. If Ansible can use it, why can't we? I don't think there is something with "fixing" Ansible, it's not a bug, this is a useful extension. What regards the mentioned action plugin for podman containers, it allows to spawn containers remotely while skipping the connection part for every cycle. I'm not sure you can "fix" Ansible not to do that,
not a bug. We may not see the difference in a few hosts in CI, but it might be very efficient when we deploy on 100+ hosts oro even 1000+ hosts. In order to evaluate this on bigger setups to understand its value we configured both options - to use action plugin or usual module. If better performance of action plugin will be proven, we can switch to use it, if it doesn't make a difference on bigger setups - then I think we can easily switch back to using an usual module.
Thanks
[1] https://docs.ansible.com/ansible/latest/plugins/plugins.html [2]
https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/action/tem...
On Mon, Aug 3, 2020 at 11:19 AM Bogdan Dobrelya <bdobreli@redhat.com <mailto:bdobreli@redhat.com>> wrote:
There is a trend of writing action plugins, see [0], for simple
it's things,
like just calling a module in a loop. I'm not sure that is the direction TripleO should go. If ansible is inefficient in this sort of tasks without custom python code written, we should fix ansible.
Otherwise,
what is the ultimate goal of that trend? Is that having only
action
plugins in roles and playbooks?
Please kindly asking the community to stop that, make a step back
and
reiterate with the taken approach. Thank you.
[0] https://review.opendev.org/716108
-- Best regards, Bogdan Dobrelya, Irc #bogdando
-- Best regards Sagi Shnaidman
-- Best regards, Bogdan Dobrelya, Irc #bogdando
-- Best regards, Bogdan Dobrelya, Irc #bogdando
-- Best regards Sagi Shnaidman
On Tue, Aug 4, 2020 at 5:42 AM Bogdan Dobrelya <bdobreli@redhat.com> wrote: (...)
I can understand that ansible should not be fixed for some composition tasks what require iterations and have complex logic for its "unit of work". And such ones also should be unit tested indeed. What I do not fully understand though is then what abandoning paunch for its action plugin had bought for us in the end?
Paunch was self-contained and had no external dependencies on fast-changing ansible frameworks. There was also no need for paunch to handle the ansible-specific execution strategies and nuances, like "what if that action plugin is called in async or in the check-mode?" Unit tests exited in paunch as well. It was easy to backport changes within a single code base.
So, looking back retrospectively, was rewriting paunch as an action plugin a simplification of the deployment framework? Please reply to yourself honestly. It does pretty same things but differently and added external framework. It is now also self-contained action plugin, since traditional tasks cannot be used in loops for this goal because of performance reasons.
I asked myself the same questions several times and to me the major driver around removing paunch was to move the container logic into Ansible modules which would be community supported vs paunch-runner code. The Ansible role version has also brought more plugability with other components of the framework (Ansible strategies, debugging, etc) but I don't think it was the major reason why we wrote it. The simplification aspect was mostly about removing the CLI which I don't think was needed at the end; and also the unique name thing which was a mistake and caused us many troubles as you know. To summarize, action plugins may be a good solution indeed, but perhaps
we should go back and use paunch instead of ansible? Same applies for *some* other tasks? That would also provide a balance, for action plugins, tasks and common sense.
Sagi is currently working on replacing the podman_containers action plugin by a module that will be able to process multiple containers at the same time, so we'll have less tasks and potentially faster operations at scale. -- Emilien Macchi
participants (5)
-
Alex Schultz
-
Bogdan Dobrelya
-
Emilien Macchi
-
Jesse Pretorius
-
Sagi Shnaidman