<div dir="ltr">Hi Bryan,<div><br></div><div>Yes, tempest can be run outside devstack deployments. Please check the README in <a href="https://github.com/openstack/tempest">https://github.com/openstack/tempest</a> on configuring tempest. </div><div><br></div><div>As in liberty, you need to copy the tests to tempest, I guess installing tempest on diff server also should work as long as congress service is discoverable (never tried though) . But just to let you know, congress Liberty version has minimal tempest coverage, In Mitaka we have enabled all the tempest tests.</div></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div>Best Regards,</div><div>Anusha</div></div></div></div></div></div>
<br><div class="gmail_quote">On 12 April 2016 at 10:43, Bryan Sullivan <span dir="ltr"><<a href="mailto:blsaws@hotmail.com" target="_blank">blsaws@hotmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><div dir="ltr"><font face="Courier New,sans-serif" color="#000000">Hi Anusha,<br><br>That helps. Just one more question: in Liberty (which I'm currently based upon) have the tempest tests been run outside of devstack deployments, i.e. in an actual OpenStack deployment? The guide you reference mentions devstack but it's not clear that the same process applies outside devstack:<br><br>e.g. "</font>To list all Congress test cases, run command in /opt/stack/tempest:<font face="Courier New,sans-serif" color="#000000">" references the "/opt/stack" folder which is not created outside of devstack environments. Thus to run them in a full OpenStack deployment, do I need to install tempest and create an "opt/stack/tempest" folder to which the tests are copied, on the same server where Congress is installed?<br><br>I'll try Mitaka soon but I expect to have the same question there: basically, are the tempest tests expected to be usable outside a devstack deploy? <br><br>I guess I could just try it, but I don't want to waste time if this is not designed to be used outside devstack environments.<br></font><br>Thanks,<br>Bryan Sullivan<br><br><div><hr>Date: Fri, 8 Apr 2016 09:01:29 +0530<br>From: <a href="mailto:anusha.iiitm@gmail.com" target="_blank">anusha.iiitm@gmail.com</a><div><div class="h5"><br>To: <a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.openstack.org</a><br>Subject: Re: [openstack-dev] [Congress] Issues with Tox testing<br><br><div dir="ltr">Hi Bryan,<div><br></div><div>tox -epy27 doesn't run tempest tests , that is tests mentioned in <a href="https://github.com/openstack/congress/tree/stable/liberty/contrib/tempest?" style="font-family:'Courier New',sans-serif;font-size:12.8px" target="_blank">https://github.com/openstack/congress/tree/stable/liberty/contrib/tempest</a> , it runs only unit tests , tests present in <a href="https://github.com/openstack/congress/tree/stable/liberty/congress/tests" target="_blank">https://github.com/openstack/congress/tree/stable/liberty/congress/tests</a> .</div><div><br></div><div>To run tempest tests, you need to manually copy the files to tempest and run the tests as mentioned in following readme <a href="https://github.com/openstack/congress/blob/stable/liberty/contrib/tempest/README.rst" target="_blank">https://github.com/openstack/congress/blob/stable/liberty/contrib/tempest/README.rst</a></div><div><br></div><div>Mitaka supports tempest plugin, so manually copying tests to tempest can be avoided if you are using mitaka.</div><div><br></div><div>Hope I clarified your question.</div><div><br></div></div><div><br clear="all"><div><div><div dir="ltr"><div><div dir="ltr"><div>Best Regards,</div><div>Anusha</div></div></div></div></div></div>
<br><div>On 8 April 2016 at 08:51, Bryan Sullivan <span dir="ltr"><<a href="mailto:blsaws@hotmail.com" target="_blank">blsaws@hotmail.com</a>></span> wrote:<br><blockquote style="border-left:1px #ccc solid;padding-left:1ex">
<div><div dir="ltr"><font face="Courier New,sans-serif" color="#000000">OK, somehow I did not pick up on that, or dropped it along the way of developing the script. Thanks for the clarification, also that Tempest is not required. I should have clarified that I'm using stable/liberty as the base. I will be moving to stable/mitaka soon, as part of the OPNFV Colorado release development.<br><br>One additional question then - are the tests run by "tox -epy27" the same as the tests in the folder <a href="https://github.com/openstack/congress/tree/stable/liberty/contrib/tempest?" target="_blank">https://github.com/openstack/congress/tree/stable/liberty/contrib/tempest?</a> If not, how are those tests supposed to be run for a non-devstack deploy (I see reference to devstack in the readme)?<br><br>I see that the folders have been reorganized for mitaka. My question is per the goal to include as much of the Congress tests as possible in the OPNFV CI/CD process. Not that I expect any to fail, I just want OPNFV to leverage the full test suite. If for liberty that's best left as the tests run by the tox command, then that's OK.</font><br><br>Thanks,<br>Bryan Sullivan<br><br><div><hr>Date: Thu, 7 Apr 2016 17:11:36 -0700<br>From: <a href="mailto:ekcs.openstack@gmail.com" target="_blank">ekcs.openstack@gmail.com</a><br>To: <a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.openstack.org</a><div><div><br>Subject: Re: [openstack-dev] [Congress] Issues with Tox testing<br><br><div><div>Thanks for the feedback, Bryan. Glad you got things working!</div></div><div><br></div><div>1. The instructions asking to install those packages are missing from kilo (we’ll fix that), but they have been there since liberty. Was it perhaps unclear because the line is too long?</div><div><ul style="padding-left:2em;color:rgb(51,51,51);font-family:'Helvetica Neue',Helvetica,'Segoe UI',Arial,freesans,sans-serif,'Apple Color Emoji','Segoe UI Emoji','Segoe UI Symbol';font-size:16px;background-color:rgb(255,255,255)"><li>Additionally:<br><pre style="overflow:auto;font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-size:14px;line-height:1.45;padding:16px;background-color:rgb(247,247,247);border-top-left-radius:3px;border-top-right-radius:3px;border-bottom-right-radius:3px;border-bottom-left-radius:3px;word-wrap:normal">$ sudo apt-get install git gcc python-dev libxml2 libxslt1-dev libzip-dev mysql-server python-mysqldb build-essential libssl-dev libffi-dev</pre></li></ul></div><div>2. Tempest should not be required by the tox tests.</div><div><br></div><div>Thanks!</div><div><br></div><span><div style="font-family:Calibri;font-size:11pt;text-align:left;color:black;BORDER-BOTTOM:medium none;BORDER-LEFT:medium none;PADDING-BOTTOM:0in;PADDING-LEFT:0in;PADDING-RIGHT:0in;BORDER-TOP:#b5c4df 1pt solid;BORDER-RIGHT:medium none;PADDING-TOP:3pt"><span style="font-weight:bold">From: </span> Bryan Sullivan <<a href="mailto:blsaws@hotmail.com" target="_blank">blsaws@hotmail.com</a>><br><span style="font-weight:bold">Reply-To: </span> "OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.openstack.org</a>><br><span style="font-weight:bold">Date: </span> Thursday, April 7, 2016 at 4:29 PM<br><span style="font-weight:bold">To: </span> "<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.openstack.org</a>" <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.openstack.org</a>><br><span style="font-weight:bold">Subject: </span> Re: [openstack-dev] [Congress] Issues with Tox testing<br></div><div><br></div><blockquote style="BORDER-LEFT:#b5c4df 5 solid;PADDING:0 0 0 5;MARGIN:0 0 0 5"><div><div><div dir="ltr"><font face="Courier New,sans-serif" color="#000000">An update: I found that there were two dependencies needed that were not clear in the guide at <a href="https://github.com/openstack/congress." target="_blank">https://github.com/openstack/congress.</a> I also installed Tempest which was not referenced before. If these additions are correct (they worked for me), they should be added to <a href="https://github.com/openstack/congress/blob/master/README.rst" target="_blank">https://github.com/openstack/congress/blob/master/README.rst</a>.<br><br>$ sudo apt-get install libffi-dev libssl-dev<br>$ cd ~/git<br>$ git clone <a href="https://github.com/openstack/tempest/" target="_blank">https://github.com/openstack/tempest/</a><br>$ cd tempest<br>$ ~/git/congress/bin/pip install -r requirements.txt<br>$ ~/git/congress/bin/pip install .<br><br>(not sure if both pip commands are needed - I'm not an expert on pip install)<br><br>After that, "tox -epy27" ran thru fine:<br><br>------------------------------------------------------------------------------------------------------- -----------<br>congress.tests.policy_engines.test_vmplacement.TestComputeVmAssignment.test_set_policy_with_dashes 27.623<br>congress.tests.policy_engines.test_vmplacement.TestComputeVmAssignment.test_set_policy 27.212<br>congress.tests.policy_engines.test_agnostic_performance.TestRuntimePerformance.test_simulate_latency 1.325<br>congress.tests.dse.test_dse.TestDSE.test_policy_tables 1.229<br>congress.tests.policy_engines.test_agnostic_performance.TestRuntimePerformance.test_select_100matches 1.184<br>congress.tests.test_congress.TestCongress.test_policy_execute 1.127<br>congress.tests.test_congress.TestCongress.test_datasource_api_model_execute 1.067<br>congress.tests.policy_engines.test_agnostic_performance.TestRuntimePerformance.test_update_nonrecursive 0.967<br>congress.tests.dse.test_dse.TestDSE.test_policy_table_publish 0.681<br>congress.tests.datasources.test_neutron_driver.TestDataSourceDriver.test_poll_subscribe 0.671<br>______________________________________________________________ summary _______________________________________________________________<br> py27: commands succeeded<br> congratulations :)<br><br></font><br><br>Thanks,<br>Bryan Sullivan<br><br><div><hr>From: <a href="mailto:blsaws@hotmail.com" target="_blank">blsaws@hotmail.com</a><br>To: <a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.openstack.org</a><br>Date: Thu, 7 Apr 2016 11:16:48 -0700<br>Subject: [openstack-dev] [Congress] Issues with Tox testing<br><br><div dir="ltr"><font face="Courier New,sans-serif" color="#000000">Hi Congress team, <br><br>A question for tox testing expert on the Congress team. I'm trying to run the tox tests as described at <a href="https://github.com/openstack/congress" target="_blank">https://github.com/openstack/congress</a>, specifically the two commands:<br><br></font>$ sudo pip install 'tox<1.7'
$ tox -epy27 <br><br><br><br>Due to conflicts with the OS-owned python config, I run these under my virtualenv created in the congress repo as:<br>$ cd ~/git/congress<br>$ bin/pip install 'tox<1.7'<br>$ bin/tox -epy27<br><br><br>But in any event (whether I try to run the tox within the virtualenv or not), I get errors such as:<br> c/_cffi_backend.c:15:17: fatal error: ffi.h: No such file or directory<br><br><br><br>What's missing in the setup for running these tests? <br><br><br><br>Note that I have all the config needed to run bash/CLI-based test scripts such as <a href="https://git.opnfv.org/cgit/copper/tree/tests/adhoc/dmz01.sh" target="_blank">https://git.opnfv.org/cgit/copper/tree/tests/adhoc/dmz01.sh</a> <br><br><br><br>Thanks,<br>Bryan Sullivan </div><br>__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: <a href="mailto:OpenStack-dev-request@lists.openstack.org" target="_blank">OpenStack-dev-request@lists.openstack.org</a>?subject:unsubscribe
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a></div> </div></div></div>__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: <a href="mailto:OpenStack-dev-request@lists.openstack.org" target="_blank">OpenStack-dev-request@lists.openstack.org</a>?subject:unsubscribe
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</blockquote></span>
<br>__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: <a href="http://lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a></div></div></div> </div></div>
<br>__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div>
<br>__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a></div></div></div> </div></div>
<br>__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div>