[openstack-dev] [Congress] Issues with Tox testing
Bryan Sullivan
blsaws at hotmail.com
Tue Apr 12 05:13:03 UTC 2016
Hi Anusha,
That helps. Just one more question: in Liberty (which I'm currently based upon) have the tempest tests been run outside of devstack deployments, i.e. in an actual OpenStack deployment? The guide you reference mentions devstack but it's not clear that the same process applies outside devstack:
e.g. "To list all Congress test cases, run command in /opt/stack/tempest:" references the "/opt/stack" folder which is not created outside of devstack environments. Thus to run them in a full OpenStack deployment, do I need to install tempest and create an "opt/stack/tempest" folder to which the tests are copied, on the same server where Congress is installed?
I'll try Mitaka soon but I expect to have the same question there: basically, are the tempest tests expected to be usable outside a devstack deploy?
I guess I could just try it, but I don't want to waste time if this is not designed to be used outside devstack environments.
Thanks,
Bryan Sullivan
Date: Fri, 8 Apr 2016 09:01:29 +0530
From: anusha.iiitm at gmail.com
To: openstack-dev at lists.openstack.org
Subject: Re: [openstack-dev] [Congress] Issues with Tox testing
Hi Bryan,
tox -epy27 doesn't run tempest tests , that is tests mentioned in https://github.com/openstack/congress/tree/stable/liberty/contrib/tempest , it runs only unit tests , tests present in https://github.com/openstack/congress/tree/stable/liberty/congress/tests .
To run tempest tests, you need to manually copy the files to tempest and run the tests as mentioned in following readme https://github.com/openstack/congress/blob/stable/liberty/contrib/tempest/README.rst
Mitaka supports tempest plugin, so manually copying tests to tempest can be avoided if you are using mitaka.
Hope I clarified your question.
Best Regards,Anusha
On 8 April 2016 at 08:51, Bryan Sullivan <blsaws at hotmail.com> wrote:
OK, somehow I did not pick up on that, or dropped it along the way of developing the script. Thanks for the clarification, also that Tempest is not required. I should have clarified that I'm using stable/liberty as the base. I will be moving to stable/mitaka soon, as part of the OPNFV Colorado release development.
One additional question then - are the tests run by "tox -epy27" the same as the tests in the folder https://github.com/openstack/congress/tree/stable/liberty/contrib/tempest? If not, how are those tests supposed to be run for a non-devstack deploy (I see reference to devstack in the readme)?
I see that the folders have been reorganized for mitaka. My question is per the goal to include as much of the Congress tests as possible in the OPNFV CI/CD process. Not that I expect any to fail, I just want OPNFV to leverage the full test suite. If for liberty that's best left as the tests run by the tox command, then that's OK.
Thanks,
Bryan Sullivan
Date: Thu, 7 Apr 2016 17:11:36 -0700
From: ekcs.openstack at gmail.com
To: openstack-dev at lists.openstack.org
Subject: Re: [openstack-dev] [Congress] Issues with Tox testing
Thanks for the feedback, Bryan. Glad you got things working!
1. The instructions asking to install those packages are missing from kilo (we’ll fix that), but they have been there since liberty. Was it perhaps unclear because the line is too long?Additionally:
$ sudo apt-get install git gcc python-dev libxml2 libxslt1-dev libzip-dev mysql-server python-mysqldb build-essential libssl-dev libffi-dev2. Tempest should not be required by the tox tests.
Thanks!
From: Bryan Sullivan <blsaws at hotmail.com>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org>
Date: Thursday, April 7, 2016 at 4:29 PM
To: "openstack-dev at lists.openstack.org" <openstack-dev at lists.openstack.org>
Subject: Re: [openstack-dev] [Congress] Issues with Tox testing
An update: I found that there were two dependencies needed that were not clear in the guide at https://github.com/openstack/congress. I also installed Tempest which was not referenced before. If these additions are correct (they worked for me), they should be added to https://github.com/openstack/congress/blob/master/README.rst.
$ sudo apt-get install libffi-dev libssl-dev
$ cd ~/git
$ git clone https://github.com/openstack/tempest/
$ cd tempest
$ ~/git/congress/bin/pip install -r requirements.txt
$ ~/git/congress/bin/pip install .
(not sure if both pip commands are needed - I'm not an expert on pip install)
After that, "tox -epy27" ran thru fine:
------------------------------------------------------------------------------------------------------- -----------
congress.tests.policy_engines.test_vmplacement.TestComputeVmAssignment.test_set_policy_with_dashes 27.623
congress.tests.policy_engines.test_vmplacement.TestComputeVmAssignment.test_set_policy 27.212
congress.tests.policy_engines.test_agnostic_performance.TestRuntimePerformance.test_simulate_latency 1.325
congress.tests.dse.test_dse.TestDSE.test_policy_tables 1.229
congress.tests.policy_engines.test_agnostic_performance.TestRuntimePerformance.test_select_100matches 1.184
congress.tests.test_congress.TestCongress.test_policy_execute 1.127
congress.tests.test_congress.TestCongress.test_datasource_api_model_execute 1.067
congress.tests.policy_engines.test_agnostic_performance.TestRuntimePerformance.test_update_nonrecursive 0.967
congress.tests.dse.test_dse.TestDSE.test_policy_table_publish 0.681
congress.tests.datasources.test_neutron_driver.TestDataSourceDriver.test_poll_subscribe 0.671
______________________________________________________________ summary _______________________________________________________________
py27: commands succeeded
congratulations :)
Thanks,
Bryan Sullivan
From: blsaws at hotmail.com
To: openstack-dev at lists.openstack.org
Date: Thu, 7 Apr 2016 11:16:48 -0700
Subject: [openstack-dev] [Congress] Issues with Tox testing
Hi Congress team,
A question for tox testing expert on the Congress team. I'm trying to run the tox tests as described at https://github.com/openstack/congress, specifically the two commands:
$ sudo pip install 'tox<1.7'
$ tox -epy27
Due to conflicts with the OS-owned python config, I run these under my virtualenv created in the congress repo as:
$ cd ~/git/congress
$ bin/pip install 'tox<1.7'
$ bin/tox -epy27
But in any event (whether I try to run the tox within the virtualenv or not), I get errors such as:
c/_cffi_backend.c:15:17: fatal error: ffi.h: No such file or directory
What's missing in the setup for running these tests?
Note that I have all the config needed to run bash/CLI-based test scripts such as https://git.opnfv.org/cgit/copper/tree/tests/adhoc/dmz01.sh
Thanks,
Bryan Sullivan
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160411/8cd47045/attachment.html>
More information about the OpenStack-dev
mailing list