[openstack-dev] [Heat] stevedore plugins (and wait conditions)

Angus Salkeld angus.salkeld at RACKSPACE.COM
Wed Jul 9 21:10:58 UTC 2014


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 09/07/14 10:17, Zane Bitter wrote:
> On 08/07/14 17:13, Angus Salkeld wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On 08/07/14 09:14, Zane Bitter wrote:
>>> I see that the new client plugins are loaded using stevedore, which is
>>> great and IMO absolutely the right tool for that job. Thanks to Angus &
>>> Steve B for implementing it.
>>>
>>> Now that we have done that work, I think there are more places we can
>>> take advantage of it too - for example, we currently have competing
>>> native wait condition resource types being implemented by Jason[1] and
>>> Steve H[2] respectively, and IMHO that is a mistake. We should have
>>> *one* native wait condition resource type, one AWS-compatible one,
>>> software deployments and any custom plugin that require signalling; and
>>> they should all use a common SignalResponder implementation that would
>>> call an API that is pluggable using stevedore. (In summary, what we're
>>
>> what's wrong with using the environment for that? Just have two resources
>> and you do something like this:
>> https://github.com/openstack/heat/blob/master/etc/heat/environment.d/default.yaml#L7
> 
> It doesn't cover other things that need signals, like software 
> deployments (third-party plugin authors are also on their own). We only 
> want n implementations not n*(number of resources that use signals) 
> implementations.
> 
>>> trying to make configurable is an implementation that should be
>>> invisible to the user, not an interface that is visible to the user, and
>>> therefore the correct unit of abstraction is an API, not a resource.)
>>>
>>
>> Totally depends if we want this to be operator configurable (config file or plugin)
>> or end user configurable (use their environment to choose the implementation).
>>
>>>
>>> I just noticed, however, that there is an already-partially-implemented
>>> blueprint[3] and further pending patches[4] to use stevedore for *all*
>>> types of plugins - particularly resource plugins[5] - in Heat. I feel
>>> very strongly that stevedore is _not_ a good fit for all of those use
>>> cases. (Disclaimer: obviously I _would_ think that, since I implemented
>>> the current system instead of using stevedore for precisely that reason.)
>>
>> haha.
>>
>>>
>>> The stated benefit of switching to stevedore is that it solves issues
>>> like https://launchpad.net/bugs/1292655 that are caused by the current
>>> convoluted layout of /contrib. I think the layout stems at least in part
>>
>> I think another great reason is consistency with how all other plugins are openstack
>> are written (stevedore).
> 
> Sure, consistency is nice, sometimes even at the expense of being not 
> quite the right tool for the job. But there are limits to that trade-off.
> 
>> Also I *really* don't think we should optimize for our contrib plugins
>> but for:
>> 1) our built in plugins
>> 2) out of tree plugins
> 
> I completely agree, which is why I was surprised by this change. It 
> seems to be deprecating a system that is working well for built-in and 
> out-of-tree plugins in order to make minor improvements to how we handle 
> contrib.
> 
>>> from a misunderstanding of how the current plugin_manager works. The
>>> point of the plugin_manager is that each plugin directory does *not*
>>> have to be a Python package - it can be any directory. Modules in the
>>> directory then appear in the package heat.engine.plugins once imported.
>>> So there is no need to do what we are currently doing, creating a
>>> resources package, and then a parent package that contains the tests
>>> package as well, and then in the tests doing:
>>>
>>>     from ..resources import docker_container  ## noqa
>>>
>>> All we really need to do is throw the resources in any old directory,
>>> add that directory to the plugin_dirs list, stick the tests in any old
>>> package, and from the tests do
>>>
>>>     from heat.engine.plugins import docker_container
>>>
>>> The main reason we haven't done this seems to be to avoid having to list
>>> the various contrib plugin dirs separately. Stevedore "solves" this by
>>> forcing us to list not only each directory but each class in each module
>>> in each directory separately. The tricky part of fixing the current
>>> layout is ensuring the contrib plugin directories get added to the
>>> plugin_dirs list during the unit tests and only during the unit tests.
>>> However, I'm confident that could be fixed with no more difficulty than
>>> the stevedore changes and with far less disruption to existing operators
>>> using custom plugins.
>>>
>>> Stevedore is ideal for configuring an implementation for a small number
>>> of well known plug points. It does not appear to be ideal for managing
>>> an application like Heat that comprises a vast collection of
>>> implementations of the same interface, each bound to its own plug point.
>>>
>> I wouldn't call our resources "vast".
> 
> I count 73 in your patch, not including contrib and assuming you didn't 
> miss any ;). It's seems clear to me that we're well past the point of 
> what the Extensions API was designed for. When everything is an 
> extension you need different tools to manage it. Quantity has a quality 
> all it's own ;)
> 
>> Really, I think it works great.
> 
> As discussed on IRC yesterday, we could potentially make the "plugin" 
> the existing resource_mapping() functions (or equivalent - stevedore 
> seems to require it to be a class, not a module, for some mysterious 
> reason?), and load them using the Hooks rather than the Extensions API 
> in stevedore. That could give us the best of both worlds - getting rid 
> of our custom import code in favour of a library, while keeping the 
> definitions of resource names close to the code.

OK, I'll try switch the resources over to using Hooks.

> 
> It's still not entirely clear to me what the benefit is to users to 
> offset the cost of deprecating the current system and forcing them to 
> eventually migrate, but if there was a consensus around this approach I 
> would support it. I maintain that the Extensions API doesn't seem like a 
> great fit.
> 
>>> For example, there's a subtle difference in how plugin_manager loads
>>> external modules - by searching a list of plugin directories for Python
>>> modules - and how stevedore does it, by loading a specified module
>>> already in the Python path. The latter is great for selecting one of a
>>> number of implementations that already exist in the code, but not so
>>> great for dropping in an additional external module, which now needs to
>>> be wrapped in a package that has to be installed in the path *and*
>>> there's still a configuration file to edit. This is way harder for a
>>> packager and/or operator to set up.
>>
>> I think you have this the wrong way around.
>> With stevedore you don't need to edit a config file and with pluginmanager
>> you do if that dir isn't in the list already.
>>
>> stevedore relies on namespaces, so you add your plugins into the "heat.resources"
>> namespace and then heat will load them (no editing of config files).
>> You do *not* need to edit heat/setup.cfg to add an out of tree resource,
>> you edit your own project's setup.cfg.
>>
>> I really don't think adding setup.cfg/py is a big issue for someone that
>> is exposing new functionality for their endusers. It's not like I am
>> going to knock up a random python file toss it into a dir and forget
>> about it. More likely someone will make a little git repo with their
>> plugin and add some unit tests and have a way of updating the final
>> deployment (at a minimum a script that downloads the current version
>> and installs it).
> 
> OK, I see the disconnect here. I was thinking only about OS-level 
> packaging, but you are talking about Python packaging. Probably my bias 
> is showing here because I am really not a fan of Python packaging, but 
> that isn't a good reason to pick one approach over the other ;)
> 
>>> This approach actually precludes a number of things we know we want to
>>> do in the future - for example it would be great if the native and AWS
>>> resource plugins were distributed as separate subpackages so that "yum
>>> install heat-engine" installed only the native resources, and a separate
>>> "yum install heat-cfn-plugins" added the AWS-compatibility resources.
>>> You can't (safely) package things that way if the installation would
>>> involve editing a config file.
>>
>>
>> I'd suggest we go further than this:
>> seperate the "engine" code from the resources and make seperate repos
>> - - heat
>> - - heat-resources-native
>> - - heat-resources-aws
>> - - heat-resources-<contrib>
>>
>> Then the repo matches the packaging more naturally, and stevedore handles
>> this well too.
>>
>> If you *really* don't like this ^
> 
> I am very -2 on this, as I discussed with Steve Baker last time it came 
> up. It would make my work an order of magnitude harder.

Yeah, I understand.

> 
> If we could make them separate Python packages within a single Git repo, 
> I would be +2 on that. I don't know if that's feasible with our current 
> tooling (though I guess it's not dissimilar to what you're doing with 
> the contrib stuff in this patch series?).
> 

Another option is we could move away from "contrib" and have:
heat/resources/<builtin>/<all native resources here>
heat/resources/optional/{aws,contrib}

The ones under "optional" have their own setup.cfg and need to be packaged
(by the distro) separately.


>> We can default to "resources are loaded but not enabled by default"
>> the each distro package can drop a file into /etc/environment.d/ that
>> enables it's resources.
> 
> Sounds like a recipe for distro bugs.
> 

What I was getting at was, we should separate the mechanism for loading the
plugins and what is "enabled/visible" to the user. And that logic should
really live in the resource registry in the environment.

>>> One of the main design goals of the current resource plugins was to move
>>> the mapping from resource names to classes away from one central point
>>> (where all of the modules were imported) and place the configuration
>>> alongside the code it applies to. I am definitely not looking forward to
>>> having to go look up a config file to find out what each resource is
>>> every time I open the autoscaling module (and I do need to remind myself
>>> _every_ time I open it), to say nothing of the constant merge conflicts
>>> that we used to have to deal with when there was a central registry.
>>
>> They are grouped by name, so you will only run into an issue when someone
>> else is working on the same area as you.
>>
>>>
>>> A central registry is also problematic for operators that modify it, who
>>> will have a difficult, manual and potentially error-prone merge task to
>>> perform on the config file every time they upgrade.
>>
>> I don't see why an operator will be editing this, they should be using
>> the environment to disable plugins/rename things. You don't have to
>> touch this if you are adding your own plugin.
>>
>>>
>>> Constraints, I feel, are very similar to resources in this respect. I am
>>> less concerned about template formats, since there are so few of them...
>>> although it would be really nice to be able to install these as
>>> subpackages too, and using stevedore appears to eliminate that as an
>>> option :(

Do we want to move constraints to Hooks as well? Guessing yes, to make
it consistent.

>>>
>>> Intrinsic functions are a different story. I'm equally opposed to
>>> defining them in a config file instead of near the code that implements
>>> them, but I now think I probably made a mistake in making them pluggable
>>> at all. (The idea to make functions pluggable pre-dated the idea of
>>> making template formats pluggable.) The ability to plug in functions to
>>> existing template formats is an invitation for operators to do so, and
>>> that is a recipe for a lot of incompatible templates being released into
>>> the world with the same version string. We should probably have each
>>> template format return a hard-coded map of intrinsic functions, and
>>> allow operators to create their own subclass to return a different set,
>>> and encourage them to register said subclass with a different version
>>> string (although we couldn't stop them from overriding the built-in
>>> implementation if they really wanted).
>>>
>>>
>>> Summarising, my view of the right tool for the job at each of the
>>> various plugin interfaces:
>>>
>>>    Client plugins       stevedore
>>
>> +1
>>
>>>    Signals              stevedore
>>
>> environment ^
>>
>>>    Resources            plugin_manager
>>>    Constraints          plugin_manager
>>>    Template formats     plugin_manager or maybe stevedore
>>
>> stevedore ^ :-)
>>
>>>    Intrinsic functions  neither (should be bound to template format)
>>
>> I agree the intrinsic's partly define the template version, so should be
>> more tightly bound.
> 
> OK, I think there's consensus on the intrinsics at least :) Let's go 
> ahead and make that change.

OK

> 
> cheers,
> Zane.
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJTvbAFAAoJEFrDYBLxZjWoZIMIAJwDOuph/tB3NocXEfrN8on0
Qe+ty3NWfzkd+JvFrpvbh6H/dm144eHl5c7ReVosHWKWTrwTwcaxlHnOI0YhhxVP
q9BpY370fj5S6f+CTKM0rPZ2Ga5zKwpbPdHdjQ8b7nz6J9udUZpdly4ptvl2os5k
H41MP6vIZxSbtNQ8b+oBbQOCwR+/9GJWCYrkW2QDn/6nAYV03vPnfPWq9TsHqy8V
vSbxM6+Vjq6H4nrVSoTxZkmIvx51d48uC6K5/KZQxnj/ERZuL92+Ygpg9tIQunUa
yXTTE8iWayHjbi7Vrs/3ZFH27b0QmHqVzrJEw6VmJuXAH5/Q32ihQBpFVgBRlIo=
=b7zo
-----END PGP SIGNATURE-----



More information about the OpenStack-dev mailing list