[tripleo] Stable policy and tripleoclient stdout
Hey all, As some of you will know there is work going on to replace Mistral with Ansible. There is an initial spec[1] and a few in-flight patches here to create the initial playbooks. In doing this work, I hit an issue I wanted to get some input in. first the tl;dr - what is our policy on changing the output from tripleoclient? It is proving difficult to keep it the same and in some cases changes will be required (we won't have Mistral execution IDs to print for example). As a quick reminder, in the current code we print from tripleoclient and Mistral sends messages via Zaqar to tripleoclient (which it then prints). This allows for "real time" updates from workflows. For example, introspection will print updates as introspection of nodes is completed. With Ansible it is tricky to have this same result. I can think of three options; 1. We run Ansible in the background, essentially hiding it and then polling OpenStack services to look for the expected state changes. We can then display this to the user. 2. We go for a "ansible native" approach and stream the ansible output to the user. This will be familiar to anyone familiar with Ansible but it will mean the output completely changes. This is also the easiest option (from an implementation point of view) 3. I have not tested this idea, but I think we could have a custom Ansible module that writes messages to a tempfile. tripleoclient could then consume them and display them to the user. This would be similar to idea 1, but rather than polling services we constantly read a file and display those "messages" from ansible. This would be closest to the current Mistral and Zaqar solution. Personally I am a bit torn. One of the reasons we want to use Ansible is because developers/users are more familiar with debugging it. However, if we hide ansible that might not help very much. So I think in the long run option 2 might be best. However, that completely changes the output, it limits us to what Ansible can output and frankly, in my experience, Ansible output is ugly and often hard to read. I am curious to know what y'all think and hopefully there are some other options too. Thanks, Dougal [1] https://review.opendev.org/#/c/679272/ [2] https://review.opendev.org/#/q/status:open+topic:mistral_to_ansible
On Mon, Oct 28, 2019 at 3:18 AM Dougal Matthews <dougal@redhat.com> wrote:
Hey all,
As some of you will know there is work going on to replace Mistral with Ansible. There is an initial spec[1] and a few in-flight patches here to create the initial playbooks. In doing this work, I hit an issue I wanted to get some input in.
first the tl;dr - what is our policy on changing the output from tripleoclient? It is proving difficult to keep it the same and in some cases changes will be required (we won't have Mistral execution IDs to print for example).
I don't think for the deployment commands that we have any expectation of a specific format. There was a large change during the Rocky timeframe when we switched over to the ansible driven deployment. That being said there have been requests to improve the output logging for the deployment as we currently use print rather than the proper logging module. This means that --log-file doesn't actually capture any of the deployment output (not ideal).
As a quick reminder, in the current code we print from tripleoclient and Mistral sends messages via Zaqar to tripleoclient (which it then prints). This allows for "real time" updates from workflows. For example, introspection will print updates as introspection of nodes is completed. With Ansible it is tricky to have this same result.
I can think of three options;
1. We run Ansible in the background, essentially hiding it and then polling OpenStack services to look for the expected state changes. We can then display this to the user.
Let's not do this since the output is already presented to the user for the undercloud/overcloud today.
2. We go for a "ansible native" approach and stream the ansible output to the user. This will be familiar to anyone familiar with Ansible but it will mean the output completely changes. This is also the easiest option (from an implementation point of view)
We already do this for the `openstack tripleo deploy` command so I would use that as it's likely the simplest and more closely resembles what we have today.
3. I have not tested this idea, but I think we could have a custom Ansible module that writes messages to a tempfile. tripleoclient could then consume them and display them to the user. This would be similar to idea 1, but rather than polling services we constantly read a file and display those "messages" from ansible. This would be closest to the current Mistral and Zaqar solution.
Sounds overly complex and prone to failures.
Personally I am a bit torn. One of the reasons we want to use Ansible is because developers/users are more familiar with debugging it. However, if we hide ansible that might not help very much. So I think in the long run option 2 might be best. However, that completely changes the output, it limits us to what Ansible can output and frankly, in my experience, Ansible output is ugly and often hard to read.
I am curious to know what y'all think and hopefully there are some other options too.
Thanks, Dougal
[1] https://review.opendev.org/#/c/679272/ [2] https://review.opendev.org/#/q/status:open+topic:mistral_to_ansible
On Mon, Oct 28, 2019 at 10:52 AM Alex Schultz <aschultz@redhat.com> wrote:
On Mon, Oct 28, 2019 at 3:18 AM Dougal Matthews <dougal@redhat.com> wrote:
Hey all,
As some of you will know there is work going on to replace Mistral with Ansible. There is an initial spec[1] and a few in-flight patches here to create the initial playbooks. In doing this work, I hit an issue I wanted to get some input in.
first the tl;dr - what is our policy on changing the output from tripleoclient? It is proving difficult to keep it the same and in some cases changes will be required (we won't have Mistral execution IDs to print for example).
I don't think for the deployment commands that we have any expectation of a specific format. There was a large change during the Rocky timeframe when we switched over to the ansible driven deployment. That being said there have been requests to improve the output logging for the deployment as we currently use print rather than the proper logging module. This means that --log-file doesn't actually capture any of the deployment output (not ideal).
As a quick reminder, in the current code we print from tripleoclient and Mistral sends messages via Zaqar to tripleoclient (which it then prints). This allows for "real time" updates from workflows. For example, introspection will print updates as introspection of nodes is completed. With Ansible it is tricky to have this same result.
I can think of three options;
1. We run Ansible in the background, essentially hiding it and then polling OpenStack services to look for the expected state changes. We can then display this to the user.
Let's not do this since the output is already presented to the user for the undercloud/overcloud today.
2. We go for a "ansible native" approach and stream the ansible output to the user. This will be familiar to anyone familiar with Ansible but it will mean the output completely changes. This is also the easiest option (from an implementation point of view)
We already do this for the `openstack tripleo deploy` command so I would use that as it's likely the simplest and more closely resembles what we have today.
+1 John
3. I have not tested this idea, but I think we could have a custom Ansible module that writes messages to a tempfile. tripleoclient could then consume them and display them to the user. This would be similar to idea 1, but rather than polling services we constantly read a file and display those "messages" from ansible. This would be closest to the current Mistral and Zaqar solution.
Sounds overly complex and prone to failures.
Personally I am a bit torn. One of the reasons we want to use Ansible is because developers/users are more familiar with debugging it. However, if we hide ansible that might not help very much. So I think in the long run option 2 might be best. However, that completely changes the output, it limits us to what Ansible can output and frankly, in my experience, Ansible output is ugly and often hard to read.
I am curious to know what y'all think and hopefully there are some other options too.
Thanks, Dougal
[1] https://review.opendev.org/#/c/679272/ [2] https://review.opendev.org/#/q/status:open+topic:mistral_to_ansible
On Mon, 28 Oct 2019 at 14:50, Alex Schultz <aschultz@redhat.com> wrote:
On Mon, Oct 28, 2019 at 3:18 AM Dougal Matthews <dougal@redhat.com> wrote:
Hey all,
As some of you will know there is work going on to replace Mistral with Ansible. There is an initial spec[1] and a few in-flight patches here to create the initial playbooks. In doing this work, I hit an issue I wanted to get some input in.
first the tl;dr - what is our policy on changing the output from tripleoclient? It is proving difficult to keep it the same and in some cases changes will be required (we won't have Mistral execution IDs to print for example).
I don't think for the deployment commands that we have any expectation of a specific format. There was a large change during the Rocky timeframe when we switched over to the ansible driven deployment. That being said there have been requests to improve the output logging for the deployment as we currently use print rather than the proper logging module. This means that --log-file doesn't actually capture any of the deployment output (not ideal).
As a quick reminder, in the current code we print from tripleoclient and Mistral sends messages via Zaqar to tripleoclient (which it then prints). This allows for "real time" updates from workflows. For example, introspection will print updates as introspection of nodes is completed. With Ansible it is tricky to have this same result.
I can think of three options;
1. We run Ansible in the background, essentially hiding it and then polling OpenStack services to look for the expected state changes. We can then display this to the user.
Let's not do this since the output is already presented to the user for the undercloud/overcloud today.
2. We go for a "ansible native" approach and stream the ansible output to the user. This will be familiar to anyone familiar with Ansible but it will mean the output completely changes. This is also the easiest option (from an implementation point of view)
We already do this for the `openstack tripleo deploy` command so I would use that as it's likely the simplest and more closely resembles what we have today.
Great. That sounds good to me. I was worried there would be more pushback about completely changing the output but I think this is the best option otherwise.
3. I have not tested this idea, but I think we could have a custom Ansible module that writes messages to a tempfile. tripleoclient could then consume them and display them to the user. This would be similar to idea 1, but rather than polling services we constantly read a file and display those "messages" from ansible. This would be closest to the current Mistral and Zaqar solution.
Sounds overly complex and prone to failures.
Personally I am a bit torn. One of the reasons we want to use Ansible is because developers/users are more familiar with debugging it. However, if we hide ansible that might not help very much. So I think in the long run option 2 might be best. However, that completely changes the output, it limits us to what Ansible can output and frankly, in my experience, Ansible output is ugly and often hard to read.
I am curious to know what y'all think and hopefully there are some other options too.
Thanks, Dougal
[1] https://review.opendev.org/#/c/679272/ [2] https://review.opendev.org/#/q/status:open+topic:mistral_to_ansible
participants (3)
-
Alex Schultz
-
Dougal Matthews
-
John Fulton