Open Stack

I've been working to add ceilometer checks in javelin2. Doing so has
revealed some issues that appear to be a fairly big deal but I suppose
there's some chance I'm doing things completely wrong.

For reference my experiments are being done with a devstack with
ceilometer enabled, running javelin as:

     python tempest/cmd/javelin.py -m check \
         -r tempest/cmd/resources.yaml

replace "check" with "create" as required.

First thing I noticed: setting sys.excepthook in setup() in
tempest/openstack/common/log.py is causing exceptions to be swallowed
such that when making simple runs it is not obvious that things have
gone wrong. You can check $? and then look in templest.log but the
content of tempest.log is just the exception message, not its type nor
any traceback. If you wish to follow along at home comment out line 427
in tempest/openstack/common/log.py.

Second thing: When run as above the path to image files is
insufficient for them to be loaded. I overcame this by hardcoding a
BASENAME (see my review in progress[1]). Note that because of the
swallowed exceptions you can run (in create or check) and not realize
that no image files were found. The code silently exits.

Third thing: Much of the above could still work if there were a
different resources.yaml or the PWD was set specifically for test runs.
However, this patchset[2] adds support for checking creating and
attaching volumes. Assuming it is expected to use the the volumes API
under tempest/services some of the calls are being made with the wrong
number of arguments (client.volumes.{create_volume,atach_volume}). Again
these errors aren't obvious because the exceptions are swallowed.

I can provide fixes for all this stuff but I wanted to first confirm
that I'm not doing something incorrectly or missing something obvious.

Some questions:

* When javelin will be run officially as part of the tests, what is
   the PWD, such that we can create an accurate path to the image
   files?
* Is the exception swallowing intentional?
* When run in grenade will javelin have any knowledge of whether the
   current check run is happening before or after the upgrade stage?

Thanks for any help and input. I'm on IRC as cdent if you want to find me
there rather than respond here.

[1] https://review.openstack.org/#/c/102354/
[2] https://review.openstack.org/#/c/100105/

-- 
Chris Dent tw:@anticdent freenode:cdent
https://tank.peermore.com/tanks/cdent