Open Stack

Mistral doesn't have to be married to RabbitMQ, there's a ZeroMQ
driver in oslo.messaging, so in theory Mistral should be able to make
use of that.

On Mon, Jun 16, 2014 at 1:42 AM, Vladimir Kozhukalov
<vkozhukalov at mirantis.com> wrote:
> Guys,
>
> First of all we need to agree about what orchestration is. In terms of Fuel
> orchestration is task management (or scheduling) + task running. In other
> words an orchestrator needs to be able to get data (yaml, json, etc.) and to
> decide what to do, when and where and then do that. For task management we
> need to have a kind of logic like that is provided by Mistral. For launching
> it just needs to have a kind of transport like that is available when we use
> mcollective or saltstack or ssh.
>
> As far as I know (I did research in Saltstack a year ago), Saltstack does
> not have mature management mechanism. What it has is so called "overstate"
> mechanism which allows one to write a script for managing tasks in multiple
> node environments like "launch task-1 on node-1, then launch task-2 on
> node-2 and then launch task-3 on node-1 again". It works, but it is
> semi-manual. I mean it is exactly what we already have and call it Astute.
> The only difference is that Astute is a wrapper around Mcollective.
>
> The only advantages I see in using Saltstack instead of Mcollective is that
> it is written in Python (Mcollective still does not have python binding) and
> that it uses ZeroMQ. Maybe those advantages are not so subtle, but let's
> take a look carefully.
>
> For example, the fact that Saltstack is written in Python allows us to use
> Saltstack directly from Nailgun. But I am absolutely sure that everyone will
> agree that would be a great architectural lack. If you ask me, Nailgun has
> to use an external task management service with highly outlined API  such as
> Mistral. Mistral already has plenty of capabilities for that. Do we really
> need to implement all that stuff?
>
> ZeroMQ is a great advantage if you have thousands of nodes. It is highly
> scalaeble. It is also allows one to avoid using one additional external
> service like Rabbit. Minus one point of failure, right? On the other hand,
> it brings us into the world of Saltstack with its own bugs despite its
> maturity.
>
> Consequently, my personal preference is to concentrate on splitting puppet
> code into independent tasks and using Mistral for resolving task
> dependencies. As our transport layer we'll then be able to use whatever we
> want (Saltstack, Mcollective, bare ssh, any kind of custom implementation,
> etc.)
>
>
>
>
> Vladimir Kozhukalov
>
>
> On Fri, Jun 13, 2014 at 8:45 AM, Mike Scherbakov <mscherbakov at mirantis.com>
> wrote:
>>
>> Dmitry,
>> please read design doc attached to
>> https://blueprints.launchpad.net/fuel/+spec/fuel-library-modularization.
>> I think it can serve as a good source of requirements which we have, and
>> then we can see what tool is the best.
>>
>> Regards,
>>
>>
>>
>>
>> On Thu, Jun 12, 2014 at 12:28 PM, Vladimir Kuklin <vkuklin at mirantis.com>
>> wrote:
>>>
>>> Guys, what we really need from orchestration tool is an ability
>>> orchestrate a big amount of task accross the nodes with all the complicated
>>> dependencies, dynamic actions (e.g. what to do on failure and on success)
>>> and parallel execution including those, that can have no additional software
>>> installed somewhere deep in the user's infrastructure (e.g. we need to send
>>> a RESTful request to vCenter). And this is the usecase of our pluggable
>>> architecture. I am wondering if saltstack can do this.
>>>
>>>
>>> On Wed, Jun 11, 2014 at 9:08 PM, Sergii Golovatiuk
>>> <sgolovatiuk at mirantis.com> wrote:
>>>>
>>>> Hi,
>>>>
>>>> That would be nice to compare Ansible and Salt. They are both Python
>>>> based. Also, Ansible has pull model also. Personally, I am big fan of
>>>> Ansible because of its simplicity and speed of playbook development.
>>>>
>>>> ~Sergii
>>>>
>>>>
>>>> On Wed, Jun 11, 2014 at 1:21 PM, Dmitriy Shulyak <dshulyak at mirantis.com>
>>>> wrote:
>>>>>
>>>>> well, i dont have any comparison chart, i can work on one based on
>>>>> requirements i've provided in initial letter, but:
>>>>> i like ansible, but it is agentless, and it wont fit well in our
>>>>> current model of communication between nailgun and orchestrator
>>>>> cloudify - java based application, even if it is pluggable with other
>>>>> language bindings - we will benefit from application in python
>>>>> salt is been around for 3-4 years, and simply compare github graphs, it
>>>>> one of the most used and active projects in python community
>>>>>
>>>>> https://github.com/stackforge/mistral/graphs/contributors
>>>>> https://github.com/saltstack/salt/graphs/contributors
>>>>>
>>>>>
>>>>> On Wed, Jun 11, 2014 at 1:04 PM, Sergii Golovatiuk
>>>>> <sgolovatiuk at mirantis.com> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> There are many mature orchestration applications (Salt, Ansible,
>>>>>> Cloudify, Mistral). Is there any comparison chart? That would be nice to
>>>>>> compare them to understand the maturity level. Thanks
>>>>>>
>>>>>> ~Sergii
>>>>>>
>>>>>>
>>>>>> On Wed, Jun 11, 2014 at 12:48 PM, Dmitriy Shulyak
>>>>>> <dshulyak at mirantis.com> wrote:
>>>>>>>
>>>>>>> Actually i am proposing salt as alternative, the main reason - salt
>>>>>>> is mature, feature full orchestration solution, that is well adopted even by
>>>>>>> our internal teams
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jun 11, 2014 at 12:37 PM, Evgeniy L <eli at mirantis.com> wrote:
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> As far as I remember we wanted to replace Astute with Mistral [1],
>>>>>>>> do we really want to have some intermediate steps (I mean salt) to do it?
>>>>>>>>
>>>>>>>> [1] https://wiki.openstack.org/wiki/Mistral
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Jun 11, 2014 at 10:38 AM, Dmitriy Shulyak
>>>>>>>> <dshulyak at mirantis.com> wrote:
>>>>>>>>>
>>>>>>>>> Yes, in my opinion salt can completely replace
>>>>>>>>> astute/mcollective/rabbitmq.
>>>>>>>>> Listen and respond to the events generated by nailgun, or any other
>>>>>>>>> plugin - not a problem.
>>>>>>>>> There is already some kind of plugin for salt that adds ability to
>>>>>>>>> execute puppet on minions (agents) [1]
>>>>>>>>>
>>>>>>>>> [1]
>>>>>>>>> http://docs.saltstack.com/en/latest/ref/modules/all/salt.modules.puppet.html
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Jun 10, 2014 at 4:06 PM, Mike Scherbakov
>>>>>>>>> <mscherbakov at mirantis.com> wrote:
>>>>>>>>>>
>>>>>>>>>> Interesting stuff.
>>>>>>>>>> Do you think that we can get rid of Astute at some point being
>>>>>>>>>> purely replaced by Salt?
>>>>>>>>>> And listening for the commands from Fuel?
>>>>>>>>>>
>>>>>>>>>> Can you please clarify, does the suggested approach implies that
>>>>>>>>>> we can have both puppet & SaltStack? Even if you ever switch to anything
>>>>>>>>>> different, it is important to provide a smooth and step-by-step way for it.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Jun 9, 2014 at 6:05 AM, Dmitriy Shulyak
>>>>>>>>>> <dshulyak at mirantis.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hi folks,
>>>>>>>>>>>
>>>>>>>>>>> I know that sometime ago saltstack was evaluated to be used as
>>>>>>>>>>> orchestrator in fuel, so I've prepared some initial specification, that
>>>>>>>>>>> addresses basic points of integration, and general requirements for
>>>>>>>>>>> orchestrator.
>>>>>>>>>>>
>>>>>>>>>>> In my opinion saltstack perfectly fits our needs, and we can
>>>>>>>>>>> benefit from using mature orchestrator, that has its own community. I still
>>>>>>>>>>> dont have all the answers, but , anyway, i would like to ask all of you to
>>>>>>>>>>> start a review for specification
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> https://docs.google.com/document/d/1uOHgxM9ZT_2IdcmWvgpEfCMoV8o0Fk7BoAlsGHEoIfs/edit?usp=sharing
>>>>>>>>>>>
>>>>>>>>>>> I will place it in fuel-docs repo as soon as specification will
>>>>>>>>>>> be full enough to start POC, or if you think that spec should placed there
>>>>>>>>>>> as is, i can do it now
>>>>>>>>>>>
>>>>>>>>>>> Thank you
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> OpenStack-dev mailing list
>>>>>>>>>>> OpenStack-dev at lists.openstack.org
>>>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Mike Scherbakov
>>>>>>>>>> #mihgen
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> OpenStack-dev mailing list
>>>>>>>>>> OpenStack-dev at lists.openstack.org
>>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> OpenStack-dev mailing list
>>>>>>>>> OpenStack-dev at lists.openstack.org
>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> OpenStack-dev mailing list
>>>>>>>> OpenStack-dev at lists.openstack.org
>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> OpenStack-dev mailing list
>>>>>>> OpenStack-dev at lists.openstack.org
>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> OpenStack-dev mailing list
>>>>>> OpenStack-dev at lists.openstack.org
>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> OpenStack-dev mailing list
>>>>> OpenStack-dev at lists.openstack.org
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>
>>>
>>>
>>> --
>>> Yours Faithfully,
>>> Vladimir Kuklin,
>>> Fuel Library Tech Lead,
>>> Mirantis, Inc.
>>> +7 (495) 640-49-04
>>> +7 (926) 702-39-68
>>> Skype kuklinvv
>>> 45bk3, Vorontsovskaya Str.
>>> Moscow, Russia,
>>> www.mirantis.com
>>> www.mirantis.ru
>>> vkuklin at mirantis.com
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>>
>> --
>> Mike Scherbakov
>> #mihgen
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Dmitry Borodaenko