<div dir="ltr"><div>Hi Matthew,<br><br></div>Thanks for the reply.<br><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, May 17, 2016 at 5:33 PM, Matthew Mosesohn <span dir="ltr"><<a href="mailto:mmosesohn@mirantis.com" target="_blank">mmosesohn@mirantis.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi Simon,<br>
<br>
For 8.0 and earlier, I would deploy ElasticSearch before deploy_end<br>
and LMA collector after post_deploy_start<br>
<br></blockquote><div><br></div><div>Unfortunately this isn't possible because the final bits of the Elasticsearch configuration need to happen only once all the ES nodes have reached the cluster.<br></div><div>And I didn't find a way (with MOS8) to run this task during the deployment phase after both the primary ES and ES groups have been executed.<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
For Mitaka and Newton releases, the task graph now skips dependencies<br>
that are not found for the role being processed. Now this "requires"<br>
dependency will work that previously errored.<br></blockquote><div><br></div><div>Good to know!<br><br></div><div>Simon<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
Best Regards,<br>
Matthew Mosesohn<br>
<div class=""><div class="h5"><br>
On Tue, May 17, 2016 at 6:27 PM, Simon Pasquier <<a href="mailto:spasquier@mirantis.com">spasquier@mirantis.com</a>> wrote:<br>
> I'm resurrecting this thread because I didn't manage to find a satisfying<br>
> solution to deal with this issue.<br>
><br>
> First let me provide more context on the use case. The Elasticsearch/Kibana<br>
> and LMA collector plugins need to synchronize their deployment. Without too<br>
> many details, here is the workflow when both plugins are deployed:<br>
> 1. [Deployment] Install the Elasticsearch/Kibana primary node.<br>
> 2. [Deployment] Install the other Elasticsearch/Kibana nodes.<br>
> 3. [Post-Deployment] Configure the Elasticsearch cluster.<br>
> 4. [Post-Deployment] Install and configure the LMA collector.<br>
><br>
> Task #4 should happen after #3 so we've specified the dependency in<br>
> deployement_tasks.yaml [0] but when the Elasticsearch/Kibana plugin isn't<br>
> deployed in the same environment (which is a valid case), it fails [1] with:<br>
><br>
> Tasks 'elasticsearch-kibana-configuration, influxdb-configuration' can't be<br>
> in requires|required_for|groups|tasks for [lma-backends] because they don't<br>
> exist in the graph<br>
><br>
> To workaround this restriction, we're using 'upload_nodes_info' as an anchor<br>
> task [2][3] since it is always present in the graph but this isn't really<br>
> elegant. Any suggestion to improve this?<br>
><br>
> BR,<br>
> Simon<br>
><br>
> [0]<br>
> <a href="https://github.com/openstack/fuel-plugin-lma-collector/blob/fd9337b43b6bdae6012f421e22847a1b0307ead0/deployment_tasks.yaml#L123-L139" rel="noreferrer" target="_blank">https://github.com/openstack/fuel-plugin-lma-collector/blob/fd9337b43b6bdae6012f421e22847a1b0307ead0/deployment_tasks.yaml#L123-L139</a><br>
> [1] <a href="https://bugs.launchpad.net/lma-toolchain/+bug/1573087" rel="noreferrer" target="_blank">https://bugs.launchpad.net/lma-toolchain/+bug/1573087</a><br>
> [2]<br>
> <a href="https://github.com/openstack/fuel-plugin-lma-collector/blob/56ef5c42f4cd719958c4c2ac3fded1b08fe2b90f/deployment_tasks.yaml#L25-L37" rel="noreferrer" target="_blank">https://github.com/openstack/fuel-plugin-lma-collector/blob/56ef5c42f4cd719958c4c2ac3fded1b08fe2b90f/deployment_tasks.yaml#L25-L37</a><br>
> [3]<br>
> <a href="https://github.com/openstack/fuel-plugin-elasticsearch-kibana/blob/4c5736dadf457b693c30e20d1a2679165ae1155a/deployment_tasks.yaml#L156-L173" rel="noreferrer" target="_blank">https://github.com/openstack/fuel-plugin-elasticsearch-kibana/blob/4c5736dadf457b693c30e20d1a2679165ae1155a/deployment_tasks.yaml#L156-L173</a><br>
><br>
> On Fri, Jan 29, 2016 at 4:27 PM, Igor Kalnitsky <<a href="mailto:ikalnitsky@mirantis.com">ikalnitsky@mirantis.com</a>><br>
> wrote:<br>
>><br>
>> Hey folks,<br>
>><br>
>> Simon P. wrote:<br>
>> > 1. Run task X for plugin A (if installed).<br>
>> > 2. Run task Y for plugin B (if installed).<br>
>> > 3. Run task Z for plugin A (if installed).<br>
>><br>
>> Simon, could you please explain do you need this at the first place? I<br>
>> can imagine this case only if your two plugins are kinda dependent on<br>
>> each other. In this case, it's better to do what was said by Andrew W.<br>
>> - set 'Task Y' to require 'Task X' and that requirement will be<br>
>> satisfied anyway (even if Task X doesn't exist in the graph).<br>
>><br>
>><br>
>> Alex S. wrote:<br>
>> > Before we get rid of tasks.yaml can we provide a mechanism for plugin<br>
>> > devs could leverage to have tasks executes at specific points in the<br>
>> > deploy process.<br>
>><br>
>> Yeah, I think that may be useful sometime. However, I'd prefer to<br>
>> avoid anchor usage as much as possible. There's no guarantees that<br>
>> other plugin didn't make any destructive actions early, that breaks<br>
>> you later. Anchors is good way to resolve possible conflicts, but they<br>
>> aren't bulletproof.<br>
>><br>
>> - igor<br>
>><br>
>> On Thu, Jan 28, 2016 at 1:31 PM, Bogdan Dobrelya <<a href="mailto:bdobrelia@mirantis.com">bdobrelia@mirantis.com</a>><br>
>> wrote:<br>
>> > On 27.01.2016 14:44, Simon Pasquier wrote:<br>
>> >> Hi,<br>
>> >><br>
>> >> I see that tasks.yaml is going to be deprecated in the future MOS<br>
>> >> versions [1]. I've got one question regarding the ordering of tasks<br>
>> >> between different plugins.<br>
>> >> With tasks.yaml, it was possible to coordinate the execution of tasks<br>
>> >> between plugins without prior knowledge of which plugins were installed<br>
>> >> [2].<br>
>> >> For example, lets say we have 2 plugins: A and B. The plugins may or<br>
>> >> may<br>
>> >> not be installed in the same environment and the tasks execution should<br>
>> >> be:<br>
>> >> 1. Run task X for plugin A (if installed).<br>
>> >> 2. Run task Y for plugin B (if installed).<br>
>> >> 3. Run task Z for plugin A (if installed).<br>
>> >><br>
>> >> Right now, we can set task priorities like:<br>
>> >><br>
>> >> # tasks.yaml for plugin A<br>
>> >> - role: ['*']<br>
>> >> stage: post_deployment/1000<br>
>> >> type: puppet<br>
>> >> parameters:<br>
>> >> puppet_manifest: puppet/manifests/task_X.pp<br>
>> >> puppet_modules: puppet/modules<br>
>> >><br>
>> >> - role: ['*']<br>
>> >> stage: post_deployment/3000<br>
>> >> type: puppet<br>
>> >> parameters:<br>
>> >> puppet_manifest: puppet/manifests/task_Z.pp<br>
>> >> puppet_modules: puppet/modules<br>
>> >><br>
>> >> # tasks.yaml for plugin B<br>
>> >> - role: ['*']<br>
>> >> stage: post_deployment/2000<br>
>> >> type: puppet<br>
>> >> parameters:<br>
>> >> puppet_manifest: puppet/manifests/task_Y.pp<br>
>> >> puppet_modules: puppet/modules<br>
>> >><br>
>> >> How would it be handled without tasks.yaml?<br>
>> ><br>
>> > I created a kinda related bug [0] and submitted a patch [1] to MOS docs<br>
>> > [2] to kill some entropy on the topic of tasks schema roles versus<br>
>> > groups and using wildcards for basic and custom roles from plugins as<br>
>> > well. There is also a fancy picture to clarify things a bit. Would be<br>
>> > nice to put more details there about custom stages as well!<br>
>> ><br>
>> > If plugins are not aware of each other, they cannot be strictly ordered<br>
>> > like "to be the very last in the deployment" as one and only shall be<br>
>> > so. That is why "coordinating the execution of tasks<br>
>> > between plugins without prior knowledge of which plugins were installed"<br>
>> > looks very confusing for me. Though, maybe wildcards with the "skipped"<br>
>> > task type may help to organize things better?<br>
>> ><br>
>> > Perhaps the Fuel plugins team could answer the question better.<br>
>> ><br>
>> > [0] <a href="https://bugs.launchpad.net/fuel/+bug/1538982" rel="noreferrer" target="_blank">https://bugs.launchpad.net/fuel/+bug/1538982</a><br>
>> > [1] <a href="https://review.fuel-infra.org/16509" rel="noreferrer" target="_blank">https://review.fuel-infra.org/16509</a><br>
>> > [2]<br>
>> ><br>
>> > <a href="https://docs.mirantis.com/openstack/fuel/fuel-7.0/reference-architecture.html#task-based-deployment" rel="noreferrer" target="_blank">https://docs.mirantis.com/openstack/fuel/fuel-7.0/reference-architecture.html#task-based-deployment</a><br>
>> ><br>
>> >><br>
>> >> Regards,<br>
>> >> Simon<br>
>> >><br>
>> >> [1] <a href="https://review.openstack.org/#/c/271417/" rel="noreferrer" target="_blank">https://review.openstack.org/#/c/271417/</a><br>
>> >> [2]<br>
>> >> <a href="https://wiki.openstack.org/wiki/Fuel/Plugins#Plugins_deployment_order" rel="noreferrer" target="_blank">https://wiki.openstack.org/wiki/Fuel/Plugins#Plugins_deployment_order</a><br>
>> >><br>
>> >><br>
>> >><br>
>> >> __________________________________________________________________________<br>
>> >> OpenStack Development Mailing List (not for usage questions)<br>
>> >> Unsubscribe:<br>
>> >> <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
>> >> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
>> >><br>
>> ><br>
>> ><br>
>> > --<br>
>> > Best regards,<br>
>> > Bogdan Dobrelya,<br>
>> > Irc #bogdando<br>
>> ><br>
>> ><br>
>> > __________________________________________________________________________<br>
>> > OpenStack Development Mailing List (not for usage questions)<br>
>> > Unsubscribe:<br>
>> > <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
>> > <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
>><br>
>> __________________________________________________________________________<br>
>> OpenStack Development Mailing List (not for usage questions)<br>
>> Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
><br>
><br>
><br>
> __________________________________________________________________________<br>
> OpenStack Development Mailing List (not for usage questions)<br>
> Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
><br>
<br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</div></div></blockquote></div><br></div></div></div></div>