Open Stack

I'm resurrecting this thread because I didn't manage to find a satisfying
solution to deal with this issue.

First let me provide more context on the use case. The Elasticsearch/Kibana
and LMA collector plugins need to synchronize their deployment. Without too
many details, here is the workflow when both plugins are deployed:
1. [Deployment] Install the Elasticsearch/Kibana primary node.
2. [Deployment] Install the other Elasticsearch/Kibana nodes.
3. [Post-Deployment] Configure the Elasticsearch cluster.
4. [Post-Deployment] Install and configure the LMA collector.

Task #4 should happen after #3 so we've specified the dependency in
deployement_tasks.yaml [0] but when the Elasticsearch/Kibana plugin isn't
deployed in the same environment (which is a valid case), it fails [1] with:

Tasks 'elasticsearch-kibana-configuration, influxdb-configuration' can't be
in requires|required_for|groups|tasks for [lma-backends] because they don't
exist in the graph

To workaround this restriction, we're using 'upload_nodes_info' as an
anchor task [2][3] since it is always present in the graph but this isn't
really elegant. Any suggestion to improve this?

BR,
Simon

[0]
https://github.com/openstack/fuel-plugin-lma-collector/blob/fd9337b43b6bdae6012f421e22847a1b0307ead0/deployment_tasks.yaml#L123-L139
[1] https://bugs.launchpad.net/lma-toolchain/+bug/1573087
[2]
https://github.com/openstack/fuel-plugin-lma-collector/blob/56ef5c42f4cd719958c4c2ac3fded1b08fe2b90f/deployment_tasks.yaml#L25-L37
[3]
https://github.com/openstack/fuel-plugin-elasticsearch-kibana/blob/4c5736dadf457b693c30e20d1a2679165ae1155a/deployment_tasks.yaml#L156-L173

On Fri, Jan 29, 2016 at 4:27 PM, Igor Kalnitsky <ikalnitsky at mirantis.com>
wrote:

> Hey folks,
>
> Simon P. wrote:
> > 1. Run task X for plugin A (if installed).
> > 2. Run task Y for plugin B (if installed).
> > 3. Run task Z for plugin A (if installed).
>
> Simon, could you please explain do you need this at the first place? I
> can imagine this case only if your two plugins are kinda dependent on
> each other. In this case, it's better to do what was said by Andrew W.
> - set 'Task Y' to require 'Task X' and that requirement will be
> satisfied anyway (even if Task X doesn't exist in the graph).
>
>
> Alex S. wrote:
> > Before we get rid of tasks.yaml can we provide a mechanism for plugin
> > devs could leverage to have tasks executes at specific points in the
> > deploy process.
>
> Yeah, I think that may be useful sometime. However, I'd prefer to
> avoid anchor usage as much as possible. There's no guarantees that
> other plugin didn't make any destructive actions early, that breaks
> you later. Anchors is good way to resolve possible conflicts, but they
> aren't bulletproof.
>
> - igor
>
> On Thu, Jan 28, 2016 at 1:31 PM, Bogdan Dobrelya <bdobrelia at mirantis.com>
> wrote:
> > On 27.01.2016 14:44, Simon Pasquier wrote:
> >> Hi,
> >>
> >> I see that tasks.yaml is going to be deprecated in the future MOS
> >> versions [1]. I've got one question regarding the ordering of tasks
> >> between different plugins.
> >> With tasks.yaml, it was possible to coordinate the execution of tasks
> >> between plugins without prior knowledge of which plugins were installed
> [2].
> >> For example, lets say we have 2 plugins: A and B. The plugins may or may
> >> not be installed in the same environment and the tasks execution should
> be:
> >> 1. Run task X for plugin A (if installed).
> >> 2. Run task Y for plugin B (if installed).
> >> 3. Run task Z for plugin A (if installed).
> >>
> >> Right now, we can set task priorities like:
> >>
> >> # tasks.yaml for plugin A
> >> - role: ['*']
> >>   stage: post_deployment/1000
> >>   type: puppet
> >>   parameters:
> >>     puppet_manifest: puppet/manifests/task_X.pp
> >>     puppet_modules: puppet/modules
> >>
> >> - role: ['*']
> >>   stage: post_deployment/3000
> >>   type: puppet
> >>   parameters:
> >>     puppet_manifest: puppet/manifests/task_Z.pp
> >>     puppet_modules: puppet/modules
> >>
> >> # tasks.yaml for plugin B
> >> - role: ['*']
> >>   stage: post_deployment/2000
> >>   type: puppet
> >>   parameters:
> >>     puppet_manifest: puppet/manifests/task_Y.pp
> >>     puppet_modules: puppet/modules
> >>
> >> How would it be handled without tasks.yaml?
> >
> > I created a kinda related bug [0] and submitted a patch [1] to MOS docs
> > [2] to kill some entropy on the topic of tasks schema roles versus
> > groups and using wildcards for basic and custom roles from plugins as
> > well. There is also a fancy picture to clarify things a bit. Would be
> > nice to put more details there about custom stages as well!
> >
> > If plugins are not aware of each other, they cannot be strictly ordered
> > like "to be the very last in the deployment" as one and only shall be
> > so. That is why "coordinating the execution of tasks
> > between plugins without prior knowledge of which plugins were installed"
> > looks very confusing for me. Though, maybe wildcards with the "skipped"
> > task type may help to organize things better?
> >
> > Perhaps the Fuel plugins team could answer the question better.
> >
> > [0] https://bugs.launchpad.net/fuel/+bug/1538982
> > [1] https://review.fuel-infra.org/16509
> > [2]
> >
> https://docs.mirantis.com/openstack/fuel/fuel-7.0/reference-architecture.html#task-based-deployment
> >
> >>
> >> Regards,
> >> Simon
> >>
> >> [1] https://review.openstack.org/#/c/271417/
> >> [2]
> https://wiki.openstack.org/wiki/Fuel/Plugins#Plugins_deployment_order
> >>
> >>
> >>
> __________________________________________________________________________
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >
> >
> > --
> > Best regards,
> > Bogdan Dobrelya,
> > Irc #bogdando
> >
> >
> __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160517/c4e4e3e7/attachment.html>