Open Stack

Thanks.  I'll start making the changes.  The other transport currently
implemented at oslo.messaging is located at
https://github.com/openstack/oslo.messaging/tree/master/oslo/messaging/_drivers,
prefixed with impl.  There are quid and zmq.


On Wed, Feb 26, 2014 at 12:03 AM, Renat Akhmerov <rakhmerov at mirantis.com>wrote:

> Winson, nice job!
>
> Now it totally makes sense to me. You're good to go with this unless
> others have objections.
>
> Just one technical dummy question (sorry, I'm not yet familiar with
> oslo.messaging): at your picture you have "Transport", so what can be
> specifically except RabbitMQ?
>
> Renat Akhmerov
> @ Mirantis Inc.
>
>
>
> On 26 Feb 2014, at 14:30, Nikolay Makhotkin <nmakhotkin at mirantis.com>
> wrote:
>
> Looks good. Thanks, Winson!
>
> Renat, What do you think?
>
>
> On Wed, Feb 26, 2014 at 10:00 AM, W Chan <m4d.coder at gmail.com> wrote:
>
>> The following link is the google doc of the proposed engine/executor
>> message flow architecture.
>> https://drive.google.com/file/d/0B4TqA9lkW12PZ2dJVFRsS0pGdEU/edit?usp=sharing
>>
>> The diagram on the right is the scalable engine where one or more engine
>> sends requests over a transport to one or more executors.  The executor
>> client, transport, and executor server follows the RPC client/server
>> design pattern<https://github.com/openstack/oslo.messaging/tree/master/oslo/messaging/rpc>in oslo.messaging.
>>
>> The diagram represents the local engine.  In reality, it's following the
>> same RPC client/server design pattern.  The only difference is that it'll
>> be configured to use a fake<https://github.com/openstack/oslo.messaging/blob/master/oslo/messaging/_drivers/impl_fake.py>RPC backend driver.  The fake driver uses in process
>> queues <http://docs.python.org/2/library/queue.html#module-Queue> shared
>> between a pair of engine and executor.
>>
>> The following are the stepwise changes I will make.
>> 1) Keep the local and scalable engine structure intact.  Create the
>> Executor Client at ./mistral/engine/scalable/executor/client.py.  Create
>> the Executor Server at ./mistral/engine/scalable/executor/service.py and
>> implement the task operations under
>> ./mistral/engine/scalable/executor/executor.py.  Delete
>> ./mistral/engine/scalable/executor/executor.py.  Modify the launcher
>> ./mistral/cmd/task_executor.py.  Modify ./mistral/engine/scalable/engine.py
>> to use the Executor Client instead of sending the message directly to
>> rabbit via pika.  The sum of this is the atomic change that keeps existing
>> structure and without breaking the code.
>> 2) Remove the local engine.
>> https://blueprints.launchpad.net/mistral/+spec/mistral-inproc-executor
>> 3) Implement versioning for the engine.
>> https://blueprints.launchpad.net/mistral/+spec/mistral-engine-versioning
>> 4) Port abstract engine to use oslo.messaging and implement the engine
>> client, engine server, and modify the API layer to consume the engine
>> client.
>> https://blueprints.launchpad.net/mistral/+spec/mistral-engine-standalone-process
>> .
>>
>> Winson
>>
>>
>> On Mon, Feb 24, 2014 at 8:07 PM, Renat Akhmerov <rakhmerov at mirantis.com>wrote:
>>
>>>
>>> On 25 Feb 2014, at 02:21, W Chan <m4d.coder at gmail.com> wrote:
>>>
>>> Renat,
>>>
>>> Regarding your comments on change
>>> https://review.openstack.org/#/c/75609/, I don't think the port to
>>> oslo.messaging is just a swap from pika to oslo.messaging.  OpenStack
>>> services as I understand is usually implemented as an RPC client/server
>>> over a messaging transport.  Sync vs async calls are done via the RPC
>>> client call and cast respectively.  The messaging transport is abstracted
>>> and concrete implementation is done via drivers/plugins.  So the
>>> architecture of the executor if ported to oslo.messaging needs to include a
>>> client, a server, and a transport.  The consumer (in this case the mistral
>>> engine) instantiates an instance of the client for the executor, makes the
>>> method call to handle task, the client then sends the request over the
>>> transport to the server.  The server picks up the request from the exchange
>>> and processes the request.  If cast (async), the client side returns
>>> immediately.  If call (sync), the client side waits for a response from the
>>> server over a reply_q (a unique queue for the session in the transport).
>>>  Also, oslo.messaging allows versioning in the message. Major version
>>> change indicates API contract changes.  Minor version indicates backend
>>> changes but with API compatibility.
>>>
>>>
>>> My main concern about this patch is not related with messaging
>>> infrastructure. I believe you know better than me how it should look like.
>>> I'm mostly concerned with the way of making changes you chose. From my
>>> perspective, it's much better to make atomic changes where every changes
>>> doesn't affect too much in existing architecture. So the first step could
>>> be to change pika to oslo.messaging with minimal structural changes without
>>> introducing versioning (could be just TODO comment saying that the
>>> framework allows it and we may want to use it in the future, to be decide),
>>> without getting rid of the current engine structure (local, scalable). Some
>>> of the things in the file structure and architecture came from the
>>> decisions made by many people and we need to be careful about changing them.
>>>
>>>
>>> So, where I'm headed with this change...  I'm implementing the basic
>>> structure/scaffolding for the new executor service using oslo.messaging
>>> (default transport with rabbit).  Since the whole change will take a few
>>> rounds, I don't want to disrupt any changes that the team is making at the
>>> moment and so I'm building the structure separately.  I'm also adding
>>> versioning (v1) in the module structure to anticipate any versioning
>>> changes in the future.   I expect the change request will lead to some
>>> discussion as we are doing here.  I will migrate the core operations of the
>>> executor (handle_task, handle_task_error, do_task_action) to the server
>>> component when we agree on the architecture and switch the consumer
>>> (engine) to use the new RPC client for the executor instead of sending the
>>> message to the queue over pika.  Also, the launcher for
>>> ./mistral/cmd/task_executor.py will change as well in subsequent round.  An
>>> example launcher is here
>>> https://github.com/uhobawuhot/interceptor/blob/master/bin/interceptor-engine.
>>>  The interceptor project here is what I use to research how oslo.messaging
>>> works.  I hope this is clear. The blueprint only changes how the request
>>> and response are being transported.  It shouldn't change how the executor
>>> currently works.
>>>
>>>
>>> Please create a document describing the approach you're pursuing here. I
>>> would expect to see the main goals you want to achieve upon completion.
>>>
>>> Finally, can you clarify the difference between local vs scalable
>>> engine?  I personally do not prefer to explicitly name the engine scalable
>>> because this requirement should be in the engine by default and we do not
>>> need to explicitly state/separate that.  But if this is a roadblock for the
>>> change, I can put the scalable structure back in the change to move this
>>> forward.
>>>
>>>
>>> Separation for local and scalable implementations appeared for
>>> historical reasons because from the beginning we didn't see how it all
>>> would look like and hence we tried different approaches to implement the
>>> engine. At some point we got 2 working versions: the one that didn't
>>> distribute anything (local) and another one that could distribute tasks
>>> over task executors via asynchronous HA transport (scalable). Later on we
>>> decided to leave them both since scalable is needed by the requirements and
>>> local might be useful for demonstration purposes and testing since it
>>> doesn't require RabbitMQ to be installed. So we decided to refactor both
>>> and make them work similarly except the way they run tasks.
>>>
>>> Thanks.
>>>
>>> Renat Akhmerov
>>> @Mirantis Inc.
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> --
> Best Regards,
> Nikolay
>  _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140226/41668d05/attachment-0001.html>