Open Stack

On Tue, Jul 01, 2014 at 02:01:19PM +0100, Chris Dent wrote:
> 
> I've been working to add ceilometer checks in javelin2. Doing so has
> revealed some issues that appear to be a fairly big deal but I suppose
> there's some chance I'm doing things completely wrong.
> 
> For reference my experiments are being done with a devstack with
> ceilometer enabled, running javelin as:
> 
>     python tempest/cmd/javelin.py -m check \
>         -r tempest/cmd/resources.yaml
> 
> replace "check" with "create" as required.
> 
> First thing I noticed: setting sys.excepthook in setup() in
> tempest/openstack/common/log.py is causing exceptions to be swallowed
> such that when making simple runs it is not obvious that things have
> gone wrong. You can check $? and then look in templest.log but the
> content of tempest.log is just the exception message, not its type nor
> any traceback. If you wish to follow along at home comment out line 427
> in tempest/openstack/common/log.py.

This was a bug with oslo logging, Sean already pushed a fix for it. I have a
sync patch here:

https://review.openstack.org/#/c/103886/

In the mean-time you can easily re-enable the full tracebacks by setting both
verbose and debug logging in the tempest config file.

> 
> Second thing: When run as above the path to image files is
> insufficient for them to be loaded. I overcame this by hardcoding a
> BASENAME (see my review in progress[1]). Note that because of the
> swallowed exceptions you can run (in create or check) and not realize
> that no image files were found. The code silently exits.

Why? Looking at the code if you use a full path for the image location in the
yaml file it should just call open() on it. I can see an issue if you're using
relative paths in the yaml, which I think is probably the problem.

> 
> Third thing: Much of the above could still work if there were a
> different resources.yaml or the PWD was set specifically for test runs.
> However, this patchset[2] adds support for checking creating and
> attaching volumes. Assuming it is expected to use the the volumes API
> under tempest/services some of the calls are being made with the wrong
> number of arguments (client.volumes.{create_volume,atach_volume}). Again
> these errors aren't obvious because the exceptions are swallowed.

So if it doesn't work then it's a bug. (I haven't verified this yet) But, fixing
issues like this one at a time as they slip through review is not a real
solution. As the next step we need to get javelin greenlit as part of the
grenade job. (or add unit tests) This is the same problem we have in the rest of
tempest, where if we don't execute code as part of gating it should just be
assumed broken. (which includes javelin2 right now)

> 
> I can provide fixes for all this stuff but I wanted to first confirm
> that I'm not doing something incorrectly or missing something obvious.
> 
> Some questions:
> 
> * When javelin will be run officially as part of the tests, what is
>   the PWD, such that we can create an accurate path to the image
>   files?
> * Is the exception swallowing intentional?
> * When run in grenade will javelin have any knowledge of whether the
>   current check run is happening before or after the upgrade stage?
> 
> Thanks for any help and input. I'm on IRC as cdent if you want to find me
> there rather than respond here.

So I think the takeaway here is that we should consider javelin2 still very much
a WIP. It's only been a few weeks since it was initially merged and it's still
not stable enough so that we can gate on it. Until we are running it in some
fashion as part of normal gating then we should be hesitant to add new features
and functionality to it.

-Matt Treinish
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140701/1437278a/attachment.pgp>