Open Stack

On Tue, Apr 22, 2014 at 07:23:20AM -0400, Sean Dague wrote:
> On 04/22/2014 06:55 AM, Daniel P. Berrange wrote:
> > On Mon, Apr 21, 2014 at 10:52:39PM +0000, Daryl Walleck wrote:
> >> I nearly opened a spec for this, but I'd really like to get some
> >> feedback first. One of the challenges I've seen lately for Nova
> >> teams not using KVM or Xen (Ironic and LXC are just a few) is
> >> how to properly run the subset of Compute tests that will run for
> >> their hypervisor or driver. Regexes are what Ironic went with,
> >> but I'm not sure how well that will work long term since it's
> >> very much dependent on naming conventions. The good thing is
> >> that the capabilities for each hypervisor/driver are well defined
> >> (https://wiki.openstack.org/wiki/HypervisorSupportMatrix), so
> >> it's just a matter of how to convey that information. I see a
> >> few ways forward from here:
> >>
> >> 1.       Expand the compute_features_group config section to include
> >> all Compute actions and make sure all tests that require specific
> >> capabilities have skipIfs or raise a skipException. This options seems
> >> it would require the least work within Tempest, but the size of the
> >> config will continue to grow as more Nova actions are added.
> >>
> >> 2.       Create a new decorator class like was done with service tags
> >> that defines what drivers the test does or does not work for, and have
> >> the definitions of the different driver capabilities be referenced by
> >> the decorator. This is nice because it gets rid of the config creep,
> >> but it's also yet another decorator, which may not be desirable.
> >>
> >> I'm going to continue working through both of these possibilities,
> >> but any feedback either solution would be appreciated.
> > 
> > It strikes me that if the test suites have a problem determining
> > support status of APIs with the currently active driver, then the
> > applications using OpenStack will likely suffer the same problem.
> > Given that it'd be desirable to ensure we can solve it in a general
> > way, rather than only consider the test suite needs.
> > 
> > On the Nova side of things, I think it would be important to ensure
> > that there is a single "OperationNotSupported" exception that is
> > always raised when the API tries to exercise a feature that is not
> > available with a specific hypervisor driver. If a test case in the
> > test suite ever receives "OperationNotSupported" it could then just
> > mark that test case as "skipped" rather than having exception propagate
> > to result in a "fail".  To me the nice thing about such an approach is
> > that you do not need to ever maintain a "matrix" of supported features,
> > as the test suite would just do the right thing whenever the driver is
> > updated.
> 
> Agreed. Though I think we probably want the Nova API to be explicit
> about what parts of the API it's ok to throw a Not Supported. Because I
> don't think it's a blanket ok. On API endpoints where this is ok, we can
> convert not supported to a skip.

Yep, this ties into a discussion I recall elsewhere about specifying
exactly which parts of the Nova API are considered mandatory features
for drivers to implement.


Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|