Open Stack

Yea that's right. Tempest will be able to run all tests(tempest one or
plugin) as long as plugin is discoverable and all services tests
interact with are reachable from Tempest installed node.

In that case, you need to configure services (keystone, congress etc)
endpoints correctly. Please refer configuration in detail-
http://docs.openstack.org/developer/tempest/configuration.html

Regards
Ghanshyam Mann


On Tue, Apr 12, 2016 at 2:46 PM, Anusha Ramineni <anusha.iiitm at gmail.com> wrote:
> Hi Bryan,
>
> Yes, tempest can be run outside devstack deployments. Please check the
> README in https://github.com/openstack/tempest on configuring tempest.
>
> As in liberty, you need to copy the tests to tempest, I guess installing
> tempest on diff server also should work as long as congress service is
> discoverable (never tried though) . But just to let you know, congress
> Liberty version has minimal tempest coverage, In Mitaka we have enabled all
> the tempest tests.
>
> Best Regards,
> Anusha
>
> On 12 April 2016 at 10:43, Bryan Sullivan <blsaws at hotmail.com> wrote:
>>
>> Hi Anusha,
>>
>> That helps. Just one more question: in Liberty (which I'm currently based
>> upon) have the tempest tests been run outside of devstack deployments, i.e.
>> in an actual OpenStack deployment? The guide you reference mentions devstack
>> but it's not clear that the same process applies outside devstack:
>>
>> e.g. "To list all Congress test cases, run command in /opt/stack/tempest:"
>> references the "/opt/stack" folder which is not created outside of devstack
>> environments. Thus to run them in a full OpenStack deployment, do I need to
>> install  tempest and create an "opt/stack/tempest" folder to which the tests
>> are copied, on the same server where Congress is installed?
>>
>> I'll try Mitaka soon but I expect to have the same question there:
>> basically, are the tempest tests expected to be usable outside a devstack
>> deploy?
>>
>> I guess I could just try it, but I don't want to waste time if this is not
>> designed to be used outside devstack environments.
>>
>> Thanks,
>> Bryan Sullivan
>>
>> ________________________________
>> Date: Fri, 8 Apr 2016 09:01:29 +0530
>> From: anusha.iiitm at gmail.com
>>
>> To: openstack-dev at lists.openstack.org
>> Subject: Re: [openstack-dev] [Congress] Issues with Tox testing
>>
>> Hi Bryan,
>>
>> tox -epy27 doesn't run tempest tests , that is tests mentioned in
>> https://github.com/openstack/congress/tree/stable/liberty/contrib/tempest ,
>> it runs only unit tests , tests present in
>> https://github.com/openstack/congress/tree/stable/liberty/congress/tests .
>>
>> To run tempest tests, you need to manually copy the files to tempest and
>> run the tests as mentioned in following readme
>> https://github.com/openstack/congress/blob/stable/liberty/contrib/tempest/README.rst
>>
>> Mitaka supports tempest plugin, so manually copying tests to tempest can
>> be avoided if you are using mitaka.
>>
>> Hope I clarified your question.
>>
>>
>> Best Regards,
>> Anusha
>>
>> On 8 April 2016 at 08:51, Bryan Sullivan <blsaws at hotmail.com> wrote:
>>
>> OK, somehow I did not pick up on that, or dropped it along the way of
>> developing the script. Thanks for the clarification, also that Tempest is
>> not required. I should have clarified that I'm using stable/liberty as the
>> base. I will be moving to stable/mitaka soon, as part of the OPNFV Colorado
>> release development.
>>
>> One additional question then - are the tests run by "tox -epy27" the same
>> as the tests in the folder
>> https://github.com/openstack/congress/tree/stable/liberty/contrib/tempest?
>> If not, how are those tests supposed to be run for a non-devstack deploy (I
>> see reference to devstack in the readme)?
>>
>> I see that the folders have been reorganized for mitaka. My question is
>> per the goal to include as much of the Congress tests as possible in the
>> OPNFV CI/CD process. Not that I expect any to fail, I just want OPNFV to
>> leverage the full test suite. If for liberty that's best left as the tests
>> run by the tox command, then that's OK.
>>
>> Thanks,
>> Bryan Sullivan
>>
>> ________________________________
>> Date: Thu, 7 Apr 2016 17:11:36 -0700
>> From: ekcs.openstack at gmail.com
>> To: openstack-dev at lists.openstack.org
>>
>> Subject: Re: [openstack-dev] [Congress] Issues with Tox testing
>>
>> Thanks for the feedback, Bryan. Glad you got things working!
>>
>> 1. The instructions asking to install those packages are missing from kilo
>> (we’ll fix that), but they have been there since liberty. Was it perhaps
>> unclear because the line is too long?
>>
>> Additionally:
>>
>> $ sudo apt-get install git gcc python-dev libxml2 libxslt1-dev libzip-dev
>> mysql-server python-mysqldb build-essential libssl-dev libffi-dev
>>
>> 2. Tempest should not be required by the tox tests.
>>
>> Thanks!
>>
>> From: Bryan Sullivan <blsaws at hotmail.com>
>> Reply-To: "OpenStack Development Mailing List (not for usage questions)"
>> <openstack-dev at lists.openstack.org>
>> Date: Thursday, April 7, 2016 at 4:29 PM
>> To: "openstack-dev at lists.openstack.org"
>> <openstack-dev at lists.openstack.org>
>> Subject: Re: [openstack-dev] [Congress] Issues with Tox testing
>>
>> An update: I found that there were two dependencies needed that were not
>> clear in the guide at https://github.com/openstack/congress. I also
>> installed Tempest which was not referenced before. If these additions are
>> correct (they worked for me), they should be added to
>> https://github.com/openstack/congress/blob/master/README.rst.
>>
>> $ sudo apt-get install libffi-dev libssl-dev
>> $ cd ~/git
>> $ git clone https://github.com/openstack/tempest/
>> $ cd tempest
>> $ ~/git/congress/bin/pip install -r requirements.txt
>> $ ~/git/congress/bin/pip install .
>>
>> (not sure if both pip commands are needed - I'm not an expert on pip
>> install)
>>
>> After that, "tox -epy27" ran thru fine:
>>
>>
>> -------------------------------------------------------------------------------------------------------
>> -----------
>>
>> congress.tests.policy_engines.test_vmplacement.TestComputeVmAssignment.test_set_policy_with_dashes
>> 27.623
>>
>> congress.tests.policy_engines.test_vmplacement.TestComputeVmAssignment.test_set_policy
>> 27.212
>>
>> congress.tests.policy_engines.test_agnostic_performance.TestRuntimePerformance.test_simulate_latency
>> 1.325
>> congress.tests.dse.test_dse.TestDSE.test_policy_tables
>> 1.229
>>
>> congress.tests.policy_engines.test_agnostic_performance.TestRuntimePerformance.test_select_100matches
>> 1.184
>> congress.tests.test_congress.TestCongress.test_policy_execute
>> 1.127
>>
>> congress.tests.test_congress.TestCongress.test_datasource_api_model_execute
>> 1.067
>>
>> congress.tests.policy_engines.test_agnostic_performance.TestRuntimePerformance.test_update_nonrecursive
>> 0.967
>> congress.tests.dse.test_dse.TestDSE.test_policy_table_publish
>> 0.681
>>
>> congress.tests.datasources.test_neutron_driver.TestDataSourceDriver.test_poll_subscribe
>> 0.671
>> ______________________________________________________________ summary
>> _______________________________________________________________
>>   py27: commands succeeded
>>   congratulations :)
>>
>>
>>
>> Thanks,
>> Bryan Sullivan
>>
>> ________________________________
>> From: blsaws at hotmail.com
>> To: openstack-dev at lists.openstack.org
>> Date: Thu, 7 Apr 2016 11:16:48 -0700
>> Subject: [openstack-dev] [Congress] Issues with Tox testing
>>
>> Hi Congress team,
>>
>> A question for tox testing expert on the Congress team. I'm trying to run
>> the tox tests as described at https://github.com/openstack/congress,
>> specifically the two commands:
>>
>> $ sudo pip install 'tox<1.7' $ tox -epy27
>>
>>
>>
>> Due to conflicts with the OS-owned python config, I run these under my
>> virtualenv created in the congress repo as:
>> $ cd ~/git/congress
>> $ bin/pip install 'tox<1.7'
>> $ bin/tox -epy27
>>
>>
>> But in any event (whether I try to run the tox within the virtualenv or
>> not), I get errors such as:
>>   c/_cffi_backend.c:15:17: fatal error: ffi.h: No such file or directory
>>
>>
>>
>> What's missing in the setup for running these tests?
>>
>>
>>
>> Note that I have all the config needed to run bash/CLI-based test scripts
>> such as https://git.opnfv.org/cgit/copper/tree/tests/adhoc/dmz01.sh
>>
>>
>>
>> Thanks,
>> Bryan Sullivan
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions) Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions) Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions) Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions) Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>