Open Stack

   - I can write a method in base test to start local executor.  I will do
   that as a separate bp.
   - After the engine is made standalone, the API will communicate to the
   engine and the engine to the executor via the oslo.messaging transport.
    This means that for the "local" option, we need to start all three
   components (API, engine, and executor) on the same process.  If the long
   term goal as you stated above is to use separate launchers for these
   components, this means that the API launcher needs to duplicate all the
   logic to launch the engine and the executor. Hence, my proposal here is to
   move the logic to launch the components into a common module and either
   have a single generic launch script that launch specific components based
   on the CLI options or have separate launch scripts that reference the
   appropriate launch function from the common module.
   - The RPC client/server in oslo.messaging do not determine the
   transport.  The transport is determine via oslo.config and then given
   explicitly to the RPC client/server.
   https://github.com/stackforge/mistral/blob/master/mistral/engine/scalable/engine.py#L31and
   https://github.com/stackforge/mistral/blob/master/mistral/cmd/task_executor.py#L63are
examples for the client and server respectively.  The in process Queue
   is instantiated within this transport object from the fake driver.  For the
   "local" option, all three components need to share the same transport in
   order to have the Queue in scope. Thus, we will need some method to have
   this transport object visible to all three components and hence my proposal
   to use a global variable and a factory method.



On Tue, Mar 11, 2014 at 10:34 PM, Renat Akhmerov <rakhmerov at mirantis.com>wrote:

>
> On 12 Mar 2014, at 06:37, W Chan <m4d.coder at gmail.com> wrote:
>
> Here're the proposed changes.
> 1) Rewrite the launch script to be more generic which contains option to
> launch all components (i.e. API, engine, executor) on the same process but
> over separate threads or launch each individually.
>
>
> You mentioned test_executor.py so I think it would make sense first to
> refactor the code in there related with acquiring transport and launching
> executor. My suggestions are:
>
>    - In test base class (mistral.tests.base.BaseTest) create the new
>    method *start_local_executor()* that would deal with getting a fake
>    driver inside and all that stuff. This would be enough for tests where we
>    need to run engine and check something. start_local_executor() can be just
>    a part of setUp() method for such tests.
>    - As for the launch script I have the following thoughts:
>       - Long-term launch scripts should be different for all API, engine
>       and executor. Now API and engine start within the same process but it's
>       just a temporary solution.
>       - Launch script for engine (which is the same as API's for now)
>       should have an option *--use-local-executor* to be able to run an
>       executor along with engine itself within the same process.
>
>
> 2) Move transport to a global variables, similar to global _engine and
> then shared by the different component.
>
>
> Not sure why we need it. Can you please explain more detailed here? The
> better way would be to initialize engine and executor with transport when
> we create them. If our current structure doesn't allow this easily we
> should discuss it and change it.
>
> In mistral.engine.engine.py we now have:
>
>  def load_engine():
>     global _engine
>     module_name = cfg.CONF.engine.engine
>     module = importutils.import_module(module_name)
>     _engine = module.get_engine()
>
> As an option we could have the code that loads engine in engine launch
> script (once we decouple it from API process) so that when we call
> get_engine() we could pass in all needed configuration parameters like
> transport.
>
> 3) Modified the engine and the executor to use a factory method to get the
> global transport
>
>
> If we made a decision on #2 we won't need it.
>
>
> A side note: when we discuss things like that I really miss DI container :)
>
> Renat Akhmerov
> @ Mirantis Inc.
>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140312/678eb7cf/attachment.html>