Open Stack

Guys,

First of all we need to agree about what orchestration is. In terms of Fuel
orchestration is task management (or scheduling) + task running. In other
words an orchestrator needs to be able to get data (yaml, json, etc.) and
to decide what to do, when and where and then do that. For task management
we need to have a kind of logic like that is provided by Mistral. For
launching it just needs to have a kind of transport like that is available
when we use mcollective or saltstack or ssh.

As far as I know (I did research in Saltstack a year ago), Saltstack does
not have mature management mechanism. What it has is so called "overstate"
mechanism which allows one to write a script for managing tasks in multiple
node environments like "launch task-1 on node-1, then launch task-2 on
node-2 and then launch task-3 on node-1 again". It works, but it is
semi-manual. I mean it is exactly what we already have and call it Astute.
The only difference is that Astute is a wrapper around Mcollective.

The only advantages I see in using Saltstack instead of Mcollective is that
it is written in Python (Mcollective still does not have python binding)
and that it uses ZeroMQ. Maybe those advantages are not so subtle, but
let's take a look carefully.

For example, the fact that Saltstack is written in Python allows us to use
Saltstack directly from Nailgun. But I am absolutely sure that everyone
will agree that would be a great architectural lack. If you ask me, Nailgun
has to use an external task management service with highly outlined API
 such as Mistral. Mistral already has plenty of capabilities for that. Do
we really need to implement all that stuff?

ZeroMQ is a great advantage if you have thousands of nodes. It is highly
scalaeble. It is also allows one to avoid using one additional external
service like Rabbit. Minus one point of failure, right? On the other hand,
it brings us into the world of Saltstack with its own bugs despite its
maturity.

Consequently, my personal preference is to concentrate on splitting puppet
code into independent tasks and using Mistral for resolving task
dependencies. As our transport layer we'll then be able to use whatever we
want (Saltstack, Mcollective, bare ssh, any kind of custom implementation,
etc.)




Vladimir Kozhukalov


On Fri, Jun 13, 2014 at 8:45 AM, Mike Scherbakov <mscherbakov at mirantis.com>
wrote:

> Dmitry,
> please read design doc attached to
> https://blueprints.launchpad.net/fuel/+spec/fuel-library-modularization.
> I think it can serve as a good source of requirements which we have, and
> then we can see what tool is the best.
>
> Regards,
>
>
>
>
> On Thu, Jun 12, 2014 at 12:28 PM, Vladimir Kuklin <vkuklin at mirantis.com>
> wrote:
>
>> Guys, what we really need from orchestration tool is an ability
>> orchestrate a big amount of task accross the nodes with all the complicated
>> dependencies, dynamic actions (e.g. what to do on failure and on success)
>> and parallel execution including those, that can have no additional
>> software installed somewhere deep in the user's infrastructure (e.g. we
>> need to send a RESTful request to vCenter). And this is the usecase of our
>> pluggable architecture. I am wondering if saltstack can do this.
>>
>>
>> On Wed, Jun 11, 2014 at 9:08 PM, Sergii Golovatiuk <
>> sgolovatiuk at mirantis.com> wrote:
>>
>>> Hi,
>>>
>>> That would be nice to compare Ansible and Salt. They are both Python
>>> based. Also, Ansible has pull model also. Personally, I am big fan of
>>> Ansible because of its simplicity and speed of playbook development.
>>>
>>> ~Sergii
>>>
>>>
>>> On Wed, Jun 11, 2014 at 1:21 PM, Dmitriy Shulyak <dshulyak at mirantis.com>
>>> wrote:
>>>
>>>> well, i dont have any comparison chart, i can work on one based on
>>>> requirements i've provided in initial letter, but:
>>>> i like ansible, but it is agentless, and it wont fit well in our
>>>> current model of communication between nailgun and orchestrator
>>>> cloudify - java based application, even if it is pluggable with other
>>>> language bindings - we will benefit from application in python
>>>> salt is been around for 3-4 years, and simply compare github graphs, it
>>>> one of the most used and active projects in python community
>>>>
>>>> https://github.com/stackforge/mistral/graphs/contributors
>>>> https://github.com/saltstack/salt/graphs/contributors
>>>>
>>>>
>>>> On Wed, Jun 11, 2014 at 1:04 PM, Sergii Golovatiuk <
>>>> sgolovatiuk at mirantis.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> There are many mature orchestration applications (Salt, Ansible,
>>>>> Cloudify, Mistral). Is there any comparison chart? That would be nice to
>>>>> compare them to understand the maturity level. Thanks
>>>>>
>>>>> ~Sergii
>>>>>
>>>>>
>>>>> On Wed, Jun 11, 2014 at 12:48 PM, Dmitriy Shulyak <
>>>>> dshulyak at mirantis.com> wrote:
>>>>>
>>>>>> Actually i am proposing salt as alternative, the main reason - salt
>>>>>> is mature, feature full orchestration solution, that is well adopted even
>>>>>> by our internal teams
>>>>>>
>>>>>>
>>>>>> On Wed, Jun 11, 2014 at 12:37 PM, Evgeniy L <eli at mirantis.com> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> As far as I remember we wanted to replace Astute with Mistral [1],
>>>>>>> do we really want to have some intermediate steps (I mean salt) to do it?
>>>>>>>
>>>>>>> [1] https://wiki.openstack.org/wiki/Mistral
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jun 11, 2014 at 10:38 AM, Dmitriy Shulyak <
>>>>>>> dshulyak at mirantis.com> wrote:
>>>>>>>
>>>>>>>> Yes, in my opinion salt can completely replace
>>>>>>>> astute/mcollective/rabbitmq.
>>>>>>>> Listen and respond to the events generated by nailgun, or any other
>>>>>>>> plugin - not a problem.
>>>>>>>> There is already some kind of plugin for salt that adds ability to
>>>>>>>> execute puppet on minions (agents) [1]
>>>>>>>>
>>>>>>>> [1]
>>>>>>>> http://docs.saltstack.com/en/latest/ref/modules/all/salt.modules.puppet.html
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Jun 10, 2014 at 4:06 PM, Mike Scherbakov <
>>>>>>>> mscherbakov at mirantis.com> wrote:
>>>>>>>>
>>>>>>>>> Interesting stuff.
>>>>>>>>> Do you think that we can get rid of Astute at some point being
>>>>>>>>> purely replaced by Salt?
>>>>>>>>> And listening for the commands from Fuel?
>>>>>>>>>
>>>>>>>>> Can you please clarify, does the suggested approach implies that
>>>>>>>>> we can have both puppet & SaltStack? Even if you ever switch to anything
>>>>>>>>> different, it is important to provide a smooth and step-by-step way for it.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Jun 9, 2014 at 6:05 AM, Dmitriy Shulyak <
>>>>>>>>> dshulyak at mirantis.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi folks,
>>>>>>>>>>
>>>>>>>>>> I know that sometime ago saltstack was evaluated to be used as
>>>>>>>>>> orchestrator in fuel, so I've prepared some initial specification, that
>>>>>>>>>> addresses basic points of integration, and general requirements for
>>>>>>>>>> orchestrator.
>>>>>>>>>>
>>>>>>>>>> In my opinion saltstack perfectly fits our needs, and we can
>>>>>>>>>> benefit from using mature orchestrator, that has its own community. I still
>>>>>>>>>> dont have all the answers, but , anyway, i would like to ask all of you to
>>>>>>>>>> start a review for specification
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> https://docs.google.com/document/d/1uOHgxM9ZT_2IdcmWvgpEfCMoV8o0Fk7BoAlsGHEoIfs/edit?usp=sharing
>>>>>>>>>>
>>>>>>>>>> I will place it in fuel-docs repo as soon as specification will
>>>>>>>>>> be full enough to start POC, or if you think that spec should placed there
>>>>>>>>>> as is, i can do it now
>>>>>>>>>>
>>>>>>>>>> Thank you
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> OpenStack-dev mailing list
>>>>>>>>>> OpenStack-dev at lists.openstack.org
>>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Mike Scherbakov
>>>>>>>>> #mihgen
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> OpenStack-dev mailing list
>>>>>>>>> OpenStack-dev at lists.openstack.org
>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> OpenStack-dev mailing list
>>>>>>>> OpenStack-dev at lists.openstack.org
>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> OpenStack-dev mailing list
>>>>>>> OpenStack-dev at lists.openstack.org
>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> OpenStack-dev mailing list
>>>>>> OpenStack-dev at lists.openstack.org
>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> OpenStack-dev mailing list
>>>>> OpenStack-dev at lists.openstack.org
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>>
>> --
>> Yours Faithfully,
>> Vladimir Kuklin,
>> Fuel Library Tech Lead,
>> Mirantis, Inc.
>> +7 (495) 640-49-04
>> +7 (926) 702-39-68
>> Skype kuklinvv
>> 45bk3, Vorontsovskaya Str.
>> Moscow, Russia,
>> www.mirantis.com <http://www.mirantis.ru/>
>> www.mirantis.ru
>> vkuklin at mirantis.com
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> --
> Mike Scherbakov
> #mihgen
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140616/a57b1ac1/attachment.html>