[openstack-dev] [Heat] stevedore plugins (and wait conditions)

Steve Baker sbaker at redhat.com
Tue Jul 8 21:39:17 UTC 2014


On 09/07/14 07:08, Zane Bitter wrote:
> I see that the new client plugins are loaded using stevedore, which is
> great and IMO absolutely the right tool for that job. Thanks to Angus
> & Steve B for implementing it.
>
> Now that we have done that work, I think there are more places we can
> take advantage of it too - for example, we currently have competing
> native wait condition resource types being implemented by Jason[1] and
> Steve H[2] respectively, and IMHO that is a mistake. We should have
> *one* native wait condition resource type, one AWS-compatible one,
> software deployments and any custom plugin that require signalling;
> and they should all use a common SignalResponder implementation that
> would call an API that is pluggable using stevedore. (In summary, what
> we're trying to make configurable is an implementation that should be
> invisible to the user, not an interface that is visible to the user,
> and therefore the correct unit of abstraction is an API, not a resource.)
>
>
> I just noticed, however, that there is an
> already-partially-implemented blueprint[3] and further pending
> patches[4] to use stevedore for *all* types of plugins - particularly
> resource plugins[5] - in Heat. I feel very strongly that stevedore is
> _not_ a good fit for all of those use cases. (Disclaimer: obviously I
> _would_ think that, since I implemented the current system instead of
> using stevedore for precisely that reason.)
>
> The stated benefit of switching to stevedore is that it solves issues
> like https://launchpad.net/bugs/1292655 that are caused by the current
> convoluted layout of /contrib. I think the layout stems at least in
> part from a misunderstanding of how the current plugin_manager works.
> The point of the plugin_manager is that each plugin directory does
> *not* have to be a Python package - it can be any directory. Modules
> in the directory then appear in the package heat.engine.plugins once
> imported. So there is no need to do what we are currently doing,
> creating a resources package, and then a parent package that contains
> the tests package as well, and then in the tests doing:
>
>   from ..resources import docker_container  ## noqa
>
> All we really need to do is throw the resources in any old directory,
> add that directory to the plugin_dirs list, stick the tests in any old
> package, and from the tests do
>
>   from heat.engine.plugins import docker_container
>
> The main reason we haven't done this seems to be to avoid having to
> list the various contrib plugin dirs separately. Stevedore "solves"
> this by forcing us to list not only each directory but each class in
> each module in each directory separately. The tricky part of fixing
> the current layout is ensuring the contrib plugin directories get
> added to the plugin_dirs list during the unit tests and only during
> the unit tests. However, I'm confident that could be fixed with no
> more difficulty than the stevedore changes and with far less
> disruption to existing operators using custom plugins.
>
There is a design document for stevedore which does a good job of
covering all the options for designing a plugin system:
http://stevedore.readthedocs.org/en/latest/essays/pycon2013.html

> Stevedore is ideal for configuring an implementation for a small
> number of well known plug points. It does not appear to be ideal for
> managing an application like Heat that comprises a vast collection of
> implementations of the same interface, each bound to its own plug point.
>
Resource plugins seems to match stevedore's Extensions pattern
reasonably well
http://stevedore.readthedocs.org/en/latest/patterns_loading.html

> For example, there's a subtle difference in how plugin_manager loads
> external modules - by searching a list of plugin directories for
> Python modules - and how stevedore does it, by loading a specified
> module already in the Python path. The latter is great for selecting
> one of a number of implementations that already exist in the code, but
> not so great for dropping in an additional external module, which now
> needs to be wrapped in a package that has to be installed in the path
> *and* there's still a configuration file to edit. This is way harder
> for a packager and/or operator to set up.
>
Just dropping in a file is convenient, but maybe properly packaging
resource plugins is a discipline we should be encouraging third parties
to adopt.
> This approach actually precludes a number of things we know we want to
> do in the future - for example it would be great if the native and AWS
> resource plugins were distributed as separate subpackages so that "yum
> install heat-engine" installed only the native resources, and a
> separate "yum install heat-cfn-plugins" added the AWS-compatibility
> resources. You can't (safely) package things that way if the
> installation would involve editing a config file.
>
Yes, patching a single setup.cfg is a non-starter. We would need to do
something like move the AWS resources to contrib with their own
setup.cfg; maybe that wouldn't be such a bad thing ;)
> One of the main design goals of the current resource plugins was to
> move the mapping from resource names to classes away from one central
> point (where all of the modules were imported) and place the
> configuration alongside the code it applies to. I am definitely not
> looking forward to having to go look up a config file to find out what
> each resource is every time I open the autoscaling module (and I do
> need to remind myself _every_ time I open it), to say nothing of the
> constant merge conflicts that we used to have to deal with when there
> was a central registry.
>
On the flip side, having a single file which has all mappings is quite
nice. Many times I've thought where the heck is the code for plugin foo.

> A central registry is also problematic for operators that modify it,
> who will have a difficult, manual and potentially error-prone merge
> task to perform on the config file every time they upgrade.
>
Agreed, it would have to be managed as separately packaged software, not
patching a single setup.cfg
> Constraints, I feel, are very similar to resources in this respect. I
> am less concerned about template formats, since there are so few of
> them... although it would be really nice to be able to install these
> as subpackages too, and using stevedore appears to eliminate that as
> an option :(
>
To me constraints are more like client plugins. They make API calls and
need specific knowledge of client exceptions. That is why I have been
moving them into the client plugin modules. It would be preferable if
they used the same plugin loading mechanism too.
> Intrinsic functions are a different story. I'm equally opposed to
> defining them in a config file instead of near the code that
> implements them, but I now think I probably made a mistake in making
> them pluggable at all. (The idea to make functions pluggable pre-dated
> the idea of making template formats pluggable.) The ability to plug in
> functions to existing template formats is an invitation for operators
> to do so, and that is a recipe for a lot of incompatible templates
> being released into the world with the same version string. We should
> probably have each template format return a hard-coded map of
> intrinsic functions, and allow operators to create their own subclass
> to return a different set, and encourage them to register said
> subclass with a different version string (although we couldn't stop
> them from overriding the built-in implementation if they really wanted).
>
I'm OK with functions being less pluggable until someone comes up with
some valid use cases to make them extendable.
>
> Summarising, my view of the right tool for the job at each of the
> various plugin interfaces:
>
>  Client plugins       stevedore
+1
>  Signals              stevedore
>  Resources            plugin_manager
Worthy of further debate
>  Constraints          plugin_manager
Would prefer stevedore
>  Template formats     plugin_manager or maybe stevedore
Either/or
>  Intrinsic functions  neither (should be bound to template format)
+1
>
> At a minimum, I think this blueprint requires more discussion than it
> has heretofore been subject to. I would be interested to hear what
> anyone else thinks.
>
One issue which I've seen in contrib is a bad interaction between
PluginManager, pkg_resources and pbr (complains of missing setup.cfg
when PluginManager loads the code; I'm sure it would be fixable). It
would be nice if third parties could contribute resources, client and
constraints in a single packaged thing using only one plugin mechanism.
>
>
> [1] https://review.openstack.org/96947
> [2] https://review.openstack.org/101354
> [3] https://blueprints.launchpad.net/heat/+spec/stevedore-plugins
> [4]
> https://review.openstack.org/#/q/status:open+project:openstack/heat+branch:master+topic:stevedore-plugins,n,z
> [5] https://review.openstack.org/103044
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




More information about the OpenStack-dev mailing list