Open Stack

On Thu, Sep 29, 2016 at 03:49:27PM +0800, Bruce Tan wrote:
> Hello everyone,
> 
> I am having a problem writing/updating a test case to verify some new
> feature (in my case, the "description" field for a network).
> 
> Acoording to Tempest Coding Guide[1], I am supposed to check if the
> related feature is there by @test.requires_ext() like this:
>     @test.requires_ext(extension="standard-attr-description",
>                        service="network")
> 
> And according to the same doc,
> > When running a test that requires a certain "feature" in the target
> > cloud, if that feature is missing we should fail, because either the
> > test configuration is invalid, or the cloud is broken and the expected
> > "feature" is not there even if the cloud was configured with it.
> 
> 
> However, my patch[2] got a "-1" from Jenkins because one check
> ("gate-tempest-dsvm-neutron-full-ubuntu-trusty-liberty") failed. The
> reason for failing, I think, is just what I quoted above: the
> tempest.conf[3]  file is configured as
>   [network-feature-enabled]api_extensions = all
> which means any api_extension is supported; but the feature I am
> testing is obviously not there in Liberty, so the API doesn't accept
> "description" field, and the test case failed.
> 
> So my question is, what did I do wrong? Is there some other way
> to skip the case for older releases? Or, maybe we shouldn't use
> "api_extensions=all" (explicitly list all extensions instead, which
> takes some effort obviously) in the configuration file?

You actually did everything correctly for making sure you skip properly
if the new feature isn't present on the tempest side. It's actually a bug in
devstack you're hitting. On stable branches we're supposed to hard code the
extension list of what api extensions are available when we branch the project.
But in the case of liberty, we neglected to do this:

http://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/tempest?h=stable%2Fliberty#n430

Instead this should look something like what we did on the kilo branch:

http://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/tempest?h=stable/kilo#n413

You just need to push up a patch to devstack's stable/liberty branch that hard
codes the extensions available like we did in previous branches.  Once that's up
add a Depends-On to your commit and it should work fine.

-Matt Treinish
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160929/ac276e33/attachment.pgp>