Open Stack

Dmitri, this is a true statement. Mistral is fully abstracted out from a concrete transport.

Renat Akhmerov
@ Mirantis Inc.



On 17 Jun 2014, at 00:52, Dmitry Borodaenko <dborodaenko at mirantis.com> wrote:

> Mistral doesn't have to be married to RabbitMQ, there's a ZeroMQ
> driver in oslo.messaging, so in theory Mistral should be able to make
> use of that.
> 
> On Mon, Jun 16, 2014 at 1:42 AM, Vladimir Kozhukalov
> <vkozhukalov at mirantis.com> wrote:
>> Guys,
>> 
>> First of all we need to agree about what orchestration is. In terms of Fuel
>> orchestration is task management (or scheduling) + task running. In other
>> words an orchestrator needs to be able to get data (yaml, json, etc.) and to
>> decide what to do, when and where and then do that. For task management we
>> need to have a kind of logic like that is provided by Mistral. For launching
>> it just needs to have a kind of transport like that is available when we use
>> mcollective or saltstack or ssh.
>> 
>> As far as I know (I did research in Saltstack a year ago), Saltstack does
>> not have mature management mechanism. What it has is so called "overstate"
>> mechanism which allows one to write a script for managing tasks in multiple
>> node environments like "launch task-1 on node-1, then launch task-2 on
>> node-2 and then launch task-3 on node-1 again". It works, but it is
>> semi-manual. I mean it is exactly what we already have and call it Astute.
>> The only difference is that Astute is a wrapper around Mcollective.
>> 
>> The only advantages I see in using Saltstack instead of Mcollective is that
>> it is written in Python (Mcollective still does not have python binding) and
>> that it uses ZeroMQ. Maybe those advantages are not so subtle, but let's
>> take a look carefully.
>> 
>> For example, the fact that Saltstack is written in Python allows us to use
>> Saltstack directly from Nailgun. But I am absolutely sure that everyone will
>> agree that would be a great architectural lack. If you ask me, Nailgun has
>> to use an external task management service with highly outlined API  such as
>> Mistral. Mistral already has plenty of capabilities for that. Do we really
>> need to implement all that stuff?
>> 
>> ZeroMQ is a great advantage if you have thousands of nodes. It is highly
>> scalaeble. It is also allows one to avoid using one additional external
>> service like Rabbit. Minus one point of failure, right? On the other hand,
>> it brings us into the world of Saltstack with its own bugs despite its
>> maturity.
>> 
>> Consequently, my personal preference is to concentrate on splitting puppet
>> code into independent tasks and using Mistral for resolving task
>> dependencies. As our transport layer we'll then be able to use whatever we
>> want (Saltstack, Mcollective, bare ssh, any kind of custom implementation,
>> etc.)
>> 
>> 
>> 
>> 
>> Vladimir Kozhukalov
>> 
>> 
>> On Fri, Jun 13, 2014 at 8:45 AM, Mike Scherbakov <mscherbakov at mirantis.com>
>> wrote:
>>> 
>>> Dmitry,
>>> please read design doc attached to
>>> https://blueprints.launchpad.net/fuel/+spec/fuel-library-modularization.
>>> I think it can serve as a good source of requirements which we have, and
>>> then we can see what tool is the best.
>>> 
>>> Regards,
>>> 
>>> 
>>> 
>>> 
>>> On Thu, Jun 12, 2014 at 12:28 PM, Vladimir Kuklin <vkuklin at mirantis.com>
>>> wrote:
>>>> 
>>>> Guys, what we really need from orchestration tool is an ability
>>>> orchestrate a big amount of task accross the nodes with all the complicated
>>>> dependencies, dynamic actions (e.g. what to do on failure and on success)
>>>> and parallel execution including those, that can have no additional software
>>>> installed somewhere deep in the user's infrastructure (e.g. we need to send
>>>> a RESTful request to vCenter). And this is the usecase of our pluggable
>>>> architecture. I am wondering if saltstack can do this.
>>>> 
>>>> 
>>>> On Wed, Jun 11, 2014 at 9:08 PM, Sergii Golovatiuk
>>>> <sgolovatiuk at mirantis.com> wrote:
>>>>> 
>>>>> Hi,
>>>>> 
>>>>> That would be nice to compare Ansible and Salt. They are both Python
>>>>> based. Also, Ansible has pull model also. Personally, I am big fan of
>>>>> Ansible because of its simplicity and speed of playbook development.
>>>>> 
>>>>> ~Sergii
>>>>> 
>>>>> 
>>>>> On Wed, Jun 11, 2014 at 1:21 PM, Dmitriy Shulyak <dshulyak at mirantis.com>
>>>>> wrote:
>>>>>> 
>>>>>> well, i dont have any comparison chart, i can work on one based on
>>>>>> requirements i've provided in initial letter, but:
>>>>>> i like ansible, but it is agentless, and it wont fit well in our
>>>>>> current model of communication between nailgun and orchestrator
>>>>>> cloudify - java based application, even if it is pluggable with other
>>>>>> language bindings - we will benefit from application in python
>>>>>> salt is been around for 3-4 years, and simply compare github graphs, it
>>>>>> one of the most used and active projects in python community
>>>>>> 
>>>>>> https://github.com/stackforge/mistral/graphs/contributors
>>>>>> https://github.com/saltstack/salt/graphs/contributors
>>>>>> 
>>>>>> 
>>>>>> On Wed, Jun 11, 2014 at 1:04 PM, Sergii Golovatiuk
>>>>>> <sgolovatiuk at mirantis.com> wrote:
>>>>>>> 
>>>>>>> Hi,
>>>>>>> 
>>>>>>> There are many mature orchestration applications (Salt, Ansible,
>>>>>>> Cloudify, Mistral). Is there any comparison chart? That would be nice to
>>>>>>> compare them to understand the maturity level. Thanks
>>>>>>> 
>>>>>>> ~Sergii
>>>>>>> 
>>>>>>> 
>>>>>>> On Wed, Jun 11, 2014 at 12:48 PM, Dmitriy Shulyak
>>>>>>> <dshulyak at mirantis.com> wrote:
>>>>>>>> 
>>>>>>>> Actually i am proposing salt as alternative, the main reason - salt
>>>>>>>> is mature, feature full orchestration solution, that is well adopted even by
>>>>>>>> our internal teams
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Wed, Jun 11, 2014 at 12:37 PM, Evgeniy L <eli at mirantis.com> wrote:
>>>>>>>>> 
>>>>>>>>> Hi,
>>>>>>>>> 
>>>>>>>>> As far as I remember we wanted to replace Astute with Mistral [1],
>>>>>>>>> do we really want to have some intermediate steps (I mean salt) to do it?
>>>>>>>>> 
>>>>>>>>> [1] https://wiki.openstack.org/wiki/Mistral
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On Wed, Jun 11, 2014 at 10:38 AM, Dmitriy Shulyak
>>>>>>>>> <dshulyak at mirantis.com> wrote:
>>>>>>>>>> 
>>>>>>>>>> Yes, in my opinion salt can completely replace
>>>>>>>>>> astute/mcollective/rabbitmq.
>>>>>>>>>> Listen and respond to the events generated by nailgun, or any other
>>>>>>>>>> plugin - not a problem.
>>>>>>>>>> There is already some kind of plugin for salt that adds ability to
>>>>>>>>>> execute puppet on minions (agents) [1]
>>>>>>>>>> 
>>>>>>>>>> [1]
>>>>>>>>>> http://docs.saltstack.com/en/latest/ref/modules/all/salt.modules.puppet.html
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On Tue, Jun 10, 2014 at 4:06 PM, Mike Scherbakov
>>>>>>>>>> <mscherbakov at mirantis.com> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Interesting stuff.
>>>>>>>>>>> Do you think that we can get rid of Astute at some point being
>>>>>>>>>>> purely replaced by Salt?
>>>>>>>>>>> And listening for the commands from Fuel?
>>>>>>>>>>> 
>>>>>>>>>>> Can you please clarify, does the suggested approach implies that
>>>>>>>>>>> we can have both puppet & SaltStack? Even if you ever switch to anything
>>>>>>>>>>> different, it is important to provide a smooth and step-by-step way for it.
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> On Mon, Jun 9, 2014 at 6:05 AM, Dmitriy Shulyak
>>>>>>>>>>> <dshulyak at mirantis.com> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> Hi folks,
>>>>>>>>>>>> 
>>>>>>>>>>>> I know that sometime ago saltstack was evaluated to be used as
>>>>>>>>>>>> orchestrator in fuel, so I've prepared some initial specification, that
>>>>>>>>>>>> addresses basic points of integration, and general requirements for
>>>>>>>>>>>> orchestrator.
>>>>>>>>>>>> 
>>>>>>>>>>>> In my opinion saltstack perfectly fits our needs, and we can
>>>>>>>>>>>> benefit from using mature orchestrator, that has its own community. I still
>>>>>>>>>>>> dont have all the answers, but , anyway, i would like to ask all of you to
>>>>>>>>>>>> start a review for specification
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> https://docs.google.com/document/d/1uOHgxM9ZT_2IdcmWvgpEfCMoV8o0Fk7BoAlsGHEoIfs/edit?usp=sharing
>>>>>>>>>>>> 
>>>>>>>>>>>> I will place it in fuel-docs repo as soon as specification will
>>>>>>>>>>>> be full enough to start POC, or if you think that spec should placed there
>>>>>>>>>>>> as is, i can do it now
>>>>>>>>>>>> 
>>>>>>>>>>>> Thank you
>>>>>>>>>>>> 
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> OpenStack-dev mailing list
>>>>>>>>>>>> OpenStack-dev at lists.openstack.org
>>>>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> --
>>>>>>>>>>> Mike Scherbakov
>>>>>>>>>>> #mihgen
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> OpenStack-dev mailing list
>>>>>>>>>>> OpenStack-dev at lists.openstack.org
>>>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> _______________________________________________
>>>>>>>>>> OpenStack-dev mailing list
>>>>>>>>>> OpenStack-dev at lists.openstack.org
>>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> _______________________________________________
>>>>>>>>> OpenStack-dev mailing list
>>>>>>>>> OpenStack-dev at lists.openstack.org
>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> _______________________________________________
>>>>>>>> OpenStack-dev mailing list
>>>>>>>> OpenStack-dev at lists.openstack.org
>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> _______________________________________________
>>>>>>> OpenStack-dev mailing list
>>>>>>> OpenStack-dev at lists.openstack.org
>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> OpenStack-dev mailing list
>>>>>> OpenStack-dev at lists.openstack.org
>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>> 
>>>>> 
>>>>> 
>>>>> _______________________________________________
>>>>> OpenStack-dev mailing list
>>>>> OpenStack-dev at lists.openstack.org
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>> 
>>>> 
>>>> 
>>>> 
>>>> --
>>>> Yours Faithfully,
>>>> Vladimir Kuklin,
>>>> Fuel Library Tech Lead,
>>>> Mirantis, Inc.
>>>> +7 (495) 640-49-04
>>>> +7 (926) 702-39-68
>>>> Skype kuklinvv
>>>> 45bk3, Vorontsovskaya Str.
>>>> Moscow, Russia,
>>>> www.mirantis.com
>>>> www.mirantis.ru
>>>> vkuklin at mirantis.com
>>>> 
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>> 
>>> 
>>> 
>>> 
>>> --
>>> Mike Scherbakov
>>> #mihgen
>>> 
>>> 
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> 
>> 
>> 
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> 
> 
> 
> 
> -- 
> Dmitry Borodaenko
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev