Open Stack

I see that the new client plugins are loaded using stevedore, which is 
great and IMO absolutely the right tool for that job. Thanks to Angus & 
Steve B for implementing it.

Now that we have done that work, I think there are more places we can 
take advantage of it too - for example, we currently have competing 
native wait condition resource types being implemented by Jason[1] and 
Steve H[2] respectively, and IMHO that is a mistake. We should have 
*one* native wait condition resource type, one AWS-compatible one, 
software deployments and any custom plugin that require signalling; and 
they should all use a common SignalResponder implementation that would 
call an API that is pluggable using stevedore. (In summary, what we're 
trying to make configurable is an implementation that should be 
invisible to the user, not an interface that is visible to the user, and 
therefore the correct unit of abstraction is an API, not a resource.)


I just noticed, however, that there is an already-partially-implemented 
blueprint[3] and further pending patches[4] to use stevedore for *all* 
types of plugins - particularly resource plugins[5] - in Heat. I feel 
very strongly that stevedore is _not_ a good fit for all of those use 
cases. (Disclaimer: obviously I _would_ think that, since I implemented 
the current system instead of using stevedore for precisely that reason.)

The stated benefit of switching to stevedore is that it solves issues 
like https://launchpad.net/bugs/1292655 that are caused by the current 
convoluted layout of /contrib. I think the layout stems at least in part 
from a misunderstanding of how the current plugin_manager works. The 
point of the plugin_manager is that each plugin directory does *not* 
have to be a Python package - it can be any directory. Modules in the 
directory then appear in the package heat.engine.plugins once imported. 
So there is no need to do what we are currently doing, creating a 
resources package, and then a parent package that contains the tests 
package as well, and then in the tests doing:

   from ..resources import docker_container  ## noqa

All we really need to do is throw the resources in any old directory, 
add that directory to the plugin_dirs list, stick the tests in any old 
package, and from the tests do

   from heat.engine.plugins import docker_container

The main reason we haven't done this seems to be to avoid having to list 
the various contrib plugin dirs separately. Stevedore "solves" this by 
forcing us to list not only each directory but each class in each module 
in each directory separately. The tricky part of fixing the current 
layout is ensuring the contrib plugin directories get added to the 
plugin_dirs list during the unit tests and only during the unit tests. 
However, I'm confident that could be fixed with no more difficulty than 
the stevedore changes and with far less disruption to existing operators 
using custom plugins.

Stevedore is ideal for configuring an implementation for a small number 
of well known plug points. It does not appear to be ideal for managing 
an application like Heat that comprises a vast collection of 
implementations of the same interface, each bound to its own plug point.

For example, there's a subtle difference in how plugin_manager loads 
external modules - by searching a list of plugin directories for Python 
modules - and how stevedore does it, by loading a specified module 
already in the Python path. The latter is great for selecting one of a 
number of implementations that already exist in the code, but not so 
great for dropping in an additional external module, which now needs to 
be wrapped in a package that has to be installed in the path *and* 
there's still a configuration file to edit. This is way harder for a 
packager and/or operator to set up.

This approach actually precludes a number of things we know we want to 
do in the future - for example it would be great if the native and AWS 
resource plugins were distributed as separate subpackages so that "yum 
install heat-engine" installed only the native resources, and a separate 
"yum install heat-cfn-plugins" added the AWS-compatibility resources. 
You can't (safely) package things that way if the installation would 
involve editing a config file.

One of the main design goals of the current resource plugins was to move 
the mapping from resource names to classes away from one central point 
(where all of the modules were imported) and place the configuration 
alongside the code it applies to. I am definitely not looking forward to 
having to go look up a config file to find out what each resource is 
every time I open the autoscaling module (and I do need to remind myself 
_every_ time I open it), to say nothing of the constant merge conflicts 
that we used to have to deal with when there was a central registry.

A central registry is also problematic for operators that modify it, who 
will have a difficult, manual and potentially error-prone merge task to 
perform on the config file every time they upgrade.

Constraints, I feel, are very similar to resources in this respect. I am 
less concerned about template formats, since there are so few of them... 
although it would be really nice to be able to install these as 
subpackages too, and using stevedore appears to eliminate that as an 
option :(

Intrinsic functions are a different story. I'm equally opposed to 
defining them in a config file instead of near the code that implements 
them, but I now think I probably made a mistake in making them pluggable 
at all. (The idea to make functions pluggable pre-dated the idea of 
making template formats pluggable.) The ability to plug in functions to 
existing template formats is an invitation for operators to do so, and 
that is a recipe for a lot of incompatible templates being released into 
the world with the same version string. We should probably have each 
template format return a hard-coded map of intrinsic functions, and 
allow operators to create their own subclass to return a different set, 
and encourage them to register said subclass with a different version 
string (although we couldn't stop them from overriding the built-in 
implementation if they really wanted).


Summarising, my view of the right tool for the job at each of the 
various plugin interfaces:

  Client plugins       stevedore
  Signals              stevedore
  Resources            plugin_manager
  Constraints          plugin_manager
  Template formats     plugin_manager or maybe stevedore
  Intrinsic functions  neither (should be bound to template format)

At a minimum, I think this blueprint requires more discussion than it 
has heretofore been subject to. I would be interested to hear what 
anyone else thinks.

cheers,
Zane.


[1] https://review.openstack.org/96947
[2] https://review.openstack.org/101354
[3] https://blueprints.launchpad.net/heat/+spec/stevedore-plugins
[4] 
https://review.openstack.org/#/q/status:open+project:openstack/heat+branch:master+topic:stevedore-plugins,n,z
[5] https://review.openstack.org/103044