Open Stack

I see your point. I had assume that the Hypervisor support matrix was something that was blessed, but that's what I get for assuming. :-) Once there's a list of required operations, I think that would become more clear. 

That said, there are server actions right now that even KVM doesn't implement (Change Password), which was one of the reasons I first implemented feature flags in Tempest. There's also some actions that don't necessarily make sense for certain drivers (resizing a bare metal server). If we make no assumptions about the underlying driver, shouldn't these capability flags go away altogether, or stay for convenience purposes? 

Either way, I'll hold off on this idea until after the summit discussion. Thanks for the feedback!

Daryl

-----Original Message-----
From: Sean Dague [mailto:sean at dague.net] 
Sent: Tuesday, April 22, 2014 5:40 AM
To: openstack-dev at lists.openstack.org
Subject: Re: [openstack-dev] [qa] Selecting Compute tests by driver/hypervisor

On 04/21/2014 06:52 PM, Daryl Walleck wrote:
> I nearly opened a spec for this, but I’d really like to get some 
> feedback first. One of the challenges I’ve seen lately for Nova teams 
> not using KVM or Xen (Ironic and LXC are just a few) is how to 
> properly run the subset of Compute tests that will run for their 
> hypervisor or driver. Regexes are what Ironic went with, but I’m not 
> sure how well that will work long term since it’s very much dependent 
> on naming conventions. The good thing is that the capabilities for 
> each hypervisor/driver are well defined 
> (https://wiki.openstack.org/wiki/HypervisorSupportMatrix), so it’s 
> just a matter of how to convey that information. I see a few ways 
> forward from here:
> 
>  
> 
> 1.       Expand the compute_features_group config section to include all
> Compute actions and make sure all tests that require specific 
> capabilities have skipIfs or raise a skipException. This options seems 
> it would require the least work within Tempest, but the size of the 
> config will continue to grow as more Nova actions are added.
> 
> 2.       Create a new decorator class like was done with service tags
> that defines what drivers the test does or does not work for, and have 
> the definitions of the different driver capabilities be referenced by 
> the decorator. This is nice because it gets rid of the config creep, 
> but it’s also yet another decorator, which may not be desirable.
> 
>  
> 
> I’m going to continue working through both of these possibilities, but 
> any feedback either solution would be appreciated.

Ironic mostly went with regexes for expediency to get something gating before their driver actually implements the requirements for the compute API.

Nova API is Nova API, the compute driver should be irrelevant. The part that is optional is specified by extensions (at the granularity level of an extension enable/disable). Creating all the knobs that are optional for extensions is good, and we're definitely not there yet. However if an API behaves differently based on compute driver, that's a problem with that compute driver.

I realize today that we're not there yet, but we have to be headed in that direction. The diagnostics API was an instance where this was pretty bad, and meant it was in no way an API, because the client had no idea what data payload it was getting back.

	-Sean

--
Sean Dague
http://dague.net