<div dir="ltr">I recently took a peek at stevedore [1] which I think Doug has to take credit for in part through work on ceilometer. I bumped into it cause Kurt called it out in his post yesterday on Marconi [2].<div><br>
</div><div>ANYWAY, the stevedore documentation also calls out ABC's as a good model for plugins [3], so there may already be more broad traction for this approach in OpenStack. I certainly think it's worth looking into further.</div>
<div><br></div><div>-Clay</div><div><br></div><div>1. <a href="https://github.com/dreamhost/stevedore/commits/master">https://github.com/dreamhost/stevedore/commits/master</a></div><div>2. <a href="http://lists.openstack.org/pipermail/openstack-dev/2013-August/014076.html">http://lists.openstack.org/pipermail/openstack-dev/2013-August/014076.html</a></div>
<div>3. <a href="http://stevedore.readthedocs.org/en/latest/tutorial/creating_plugins.html">http://stevedore.readthedocs.org/en/latest/tutorial/creating_plugins.html</a></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">
On Wed, Aug 21, 2013 at 10:22 PM, Morgan Fainberg <span dir="ltr"><<a href="mailto:m@metacloud.com" target="_blank">m@metacloud.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">I've been doing some pondering on how Keystone handles the various pluggable systems with it's Manager / Driver architecture.<div><br></div><div>Currently we implement the base driver class as follows:</div>
<div><br></div><div>There is a driver object that has a number of reference functions defined on it, each of these functions typically raises NotImplemented() and has a docstring describing the expected behavior. A simple example of this would be the Token Driver base class. A complex example would be the Identity Driver base class.</div>
<div><br></div><div>Example:</div><div><br></div><div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><font face="courier new, monospace">class AbstractBaseClassTest(object):</font></div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px">
<div><font face="courier new, monospace"> def abstract_method1(args, kwargs):</font></div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><font face="courier new, monospace"> raise NotImplemented()</font></div>
<div><font face="courier new, monospace"><br></font></div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><font face="courier new, monospace"> def abstract_method2(args, kwargs):</font></div>
<div><font face="courier new, monospace"> ...</font></div></blockquote></div><div><br></div><div>This type of system is not inherently bad, nor flawed, but there are some cases when I've run into situations that raise a NotImplemented() error unexpectedly (usually in custom code I've had to write around a driver or replace a driver with and "internal to my company" implementation).</div>
<div><br></div><div><br></div><div>For those not familiar with ABCMeta, abstract base classes, and the abc module: </div><div><br></div><div>In a model that uses an abstract metaclass, the base class methods that need to be overridden are decorated with the "abstractmethod" decorator from the "abc" module. This decorator when coupled with the ABCMeta metaclass, requires all abstract methods to be overridden by the class that inherits from the base class before the class can be instantiated. Similarly there is an "abstractproperty" decorator for @property methods. </div>
<div><br></div><div>The exception raised looks like: </div><div><br></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><font face="courier new, monospace">TypeError: Can't instantiate abstract class TestClass with abstract methods AbsTest1, AbsTest2</font></div>
</blockquote><div><br></div><div>The benefits are two fold. First, this means that a new driver could not be implemented without overriding the expected methods (no raising "NotImplemented()" unintentionally) and it guarantees that even seldom-used expected functions would be defined on the driver implementations. Second benefit is that these abstract methods can be called via the standard super() call, therefore if there is some base functionality that should be used (or legitimately you want to raise NotImplemented()), it is possible to have that defined on the parent class.</div>
<div><br></div><div>Example abstract base class (with using six module instead of directly setting __metaclass__):</div><div><br></div><div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><font face="courier new, monospace">class AbstractBaseClassTest(six.with_metaclass(abc.ABCMeta)):<br>
@abc.abstractmethod<br> def abstract_method1(int_arg):<br> # we can do something here instead of raising<br> # NotImplemented()<br> return (int_arg + 1)</font></blockquote></div><div><br></div>
<div>On to the downsides, the compatibility between python2.7 and python3k (using the six library) is not the most pleasing thing to look at. It requires defining the object that inherits from a method call six.with_metaclass. I also have not looked at the performance implications of this, however, at first blush, it doesn't look like should be significant. There are possibly other pitfalls that haven't come to mind as of yet.</div>
<div><br></div><div>In short, the drivers should probably be actual abstract classes, since that is what they effectively are. I've seen this functionality used in both Neutron and Ironic. I could see it providing some benefits from it in Keystone. I wanted to get the general feeling from the Mailing List on using abstracted base classes in lieu of our current implementation prior to proposing a Blueprint, etc. I don't see this as a Havana implementation but possibly something to consider for Icehouse.</div>
<div><br></div><div>I don't know if the benefits really would bring us a huge win in Keystone, but I think it would make understanding what should be implemented when subclassing for driver development (and similar pluggable systems) a bit more clear.<br>
</div><div><br></div><div>Cheers!</div><div>--</div><div>Morgan Fainberg</div><div>Sr. Software Architect | Metacloud, Inc</div><div>Email: <a href="mailto:m@metacloud.com" target="_blank">m@metacloud.com</a></div><div>IRC: morganfainberg</div>
</div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div>