On Fri, Sep 28, 2018 at 02:39:24PM +0100, Chris Dent wrote: > > I'm still trying to figure out how to properly create a "modern" (as > in zuul v3 oriented) integration test for placement using gabbi and > tempest. That work is happening at https://review.openstack.org/#/c/601614/ > > There was lots of progress made after the last message on this > topic http://lists.openstack.org/pipermail/openstack-dev/2018-September/134837.html > but I've reached another interesting impasse. > > From devstack's standpoint, the way to say "I want to use a tempest > plugin" is to set TEMPEST_PLUGINS to alist of where the plugins are. > devstack:lib/tempest then does a: > > tox -evenv-tempest -- pip install -c $REQUIREMENTS_DIR/upper-constraints.txt $TEMPEST_PLUGINS > > http://logs.openstack.org/14/601614/21/check/placement-tempest-gabbi/f44c185/job-output.txt.gz#_2018-09-28_11_12_58_138163 > > I have this part working as expected. > > However, > > The advice is then to create a new job that has a parent of > devstack-tempest. That zuul job runs a variety of tox environments, > depending on the setting of the `tox_envlist` var. If you wish to > use a `tempest_test_regex` (I do) the preferred tox environment is > 'all'. > > That venv doesn't have the plugin installed, thus no gabbi tests are > found: > > http://logs.openstack.org/14/601614/21/check/placement-tempest-gabbi/f44c185/job-output.txt.gz#_2018-09-28_11_13_25_798683 Right above this line it shows that the gabbi-tempest plugin is installed in the venv: http://logs.openstack.org/14/601614/21/check/placement-tempest-gabbi/f44c185/job-output.txt.gz#_2018-09-28_11_13_25_650661 at version 0.1.1. It's a bit weird because it's line wrapped in my browser. The devstack logs also shows the plugin: http://logs.openstack.org/14/601614/21/check/placement-tempest-gabbi/f44c185/controller/logs/devstacklog.txt.gz#_2018-09-28_11_13_13_076 All the tempest tox jobs that run tempest (and the tempest-venv command used by devstack) run inside the same tox venv: https://github.com/openstack/tempest/blob/master/tox.ini#L52 My guess is that the plugin isn't returning any tests that match the regex. I'm also a bit alarmed that tempest run is returning 0 there when no tests are being run. That's definitely a bug because things should fail with no tests being successfully run. -Matt Treinish > > How do I get my plugin installed into the right venv while still > following the guidelines for good zuul behavior? > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20180928/8809c9cb/attachment.sig>