[openstack-dev] [Mistral] Local vs. Scalable Engine

Ryan Petrello ryan.petrello at dreamhost.com
Mon Mar 17 16:37:57 UTC 2014


Changing the configuration object at runtime is not thread-safe.  If you want to share objects with controllers, I’d suggest checking out Pecan’s hook functionality.

http://pecan.readthedocs.org/en/latest/hooks.html#implementating-a-pecan-hook

e.g.,

class SpecialContextHook(object):

    def __init__(self, some_obj):
        self.some_obj = some_obj

    def before(self, state):
        # In any pecan controller, `pecan.request` is a thread-local webob.Request instance,
        # allowing you to access `pecan.request.context[‘foo’]` in your controllers.  In this example,
        # self.some_obj could be just about anything - a Python primitive, or an instance of some class
        state.request.context = {
            ‘foo’: self.some_obj
        }

...

wsgi_app = pecan.Pecan(
    my_package.controllers.root.RootController(),
    hooks=[SpecialContextHook(SomeObj(1, 2, 3))]
)

---
Ryan Petrello
Senior Developer, DreamHost
ryan.petrello at dreamhost.com

On Mar 14, 2014, at 8:53 AM, Renat Akhmerov <rakhmerov at mirantis.com> wrote:

> Take a look at method get_pecan_config() in mistral/api/app.py. It’s where you can pass any parameters into pecan app (see a dictionary ‘cfg_dict’ initialization). They can be then accessed via pecan.conf as described here: http://pecan.readthedocs.org/en/latest/configuration.html#application-configuration. If I understood the problem correctly this should be helpful.
> 
> Renat Akhmerov
> @ Mirantis Inc.
> 
> 
> 
> On 14 Mar 2014, at 05:14, Dmitri Zimine <dz at stackstorm.com> wrote:
> 
>> We have access to all configuration parameters in the context of api.py. May be you don't pass it but just instantiate it where you need it? Or I may misunderstand what you're trying to do...
>> 
>> DZ> 
>> 
>> PS: can you generate and update mistral.config.example to include new oslo messaging options? I forgot to mention it on review on time. 
>> 
>> 
>> On Mar 13, 2014, at 11:15 AM, W Chan <m4d.coder at gmail.com> wrote:
>> 
>>> On the transport variable, the problem I see isn't with passing the variable to the engine and executor.  It's passing the transport into the API layer.  The API layer is a pecan app and I currently don't see a way where the transport variable can be passed to it directly.  I'm looking at https://github.com/stackforge/mistral/blob/master/mistral/cmd/api.py#L50 and https://github.com/stackforge/mistral/blob/master/mistral/api/app.py#L44.  Do you have any suggestion?  Thanks. 
>>> 
>>> 
>>> On Thu, Mar 13, 2014 at 1:30 AM, Renat Akhmerov <rakhmerov at mirantis.com> wrote:
>>> 
>>> On 13 Mar 2014, at 10:40, W Chan <m4d.coder at gmail.com> wrote:
>>> 
>>>> 	• I can write a method in base test to start local executor.  I will do that as a separate bp.  
>>> Ok.
>>> 
>>>> 	• After the engine is made standalone, the API will communicate to the engine and the engine to the executor via the oslo.messaging transport.  This means that for the "local" option, we need to start all three components (API, engine, and executor) on the same process.  If the long term goal as you stated above is to use separate launchers for these components, this means that the API launcher needs to duplicate all the logic to launch the engine and the executor. Hence, my proposal here is to move the logic to launch the components into a common module and either have a single generic launch script that launch specific components based on the CLI options or have separate launch scripts that reference the appropriate launch function from the common module.
>>> Ok, I see your point. Then I would suggest we have one script which we could use to run all the components (any subset of of them). So for those components we specified when launching the script we use this local transport. Btw, scheduler eventually should become a standalone component too, so we have 4 components.
>>> 
>>>> 	• The RPC client/server in oslo.messaging do not determine the transport.  The transport is determine via oslo.config and then given explicitly to the RPC client/server.  https://github.com/stackforge/mistral/blob/master/mistral/engine/scalable/engine.py#L31 and https://github.com/stackforge/mistral/blob/master/mistral/cmd/task_executor.py#L63 are examples for the client and server respectively.  The in process Queue is instantiated within this transport object from the fake driver.  For the "local" option, all three components need to share the same transport in order to have the Queue in scope. Thus, we will need some method to have this transport object visible to all three components and hence my proposal to use a global variable and a factory method. 
>>> I’m still not sure I follow your point here.. Looking at the links you provided I see this:
>>> 
>>> transport = messaging.get_transport(cfg.CONF)
>>> 
>>> So my point here is we can make this call once in the launching script and pass it to engine/executor (and now API too if we want it to be launched by the same script). Of course, we’ll have to change the way how we initialize these components, but I believe we can do it. So it’s just a dependency injection. And in this case we wouldn’t need to use a global variable. Am I still missing something?
>>> 
>>> 
>>> Renat Akhmerov
>>> @ Mirantis Inc.
>>> 
>>> 
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> 
>>> 
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> 
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




More information about the OpenStack-dev mailing list