<div dir="ltr"><div class="gmail_default" style="font-family:courier new,monospace;font-size:x-small"><br></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Jun 17, 2013 at 5:01 PM, Joe Gordon <span dir="ltr"><<a href="mailto:joe.gordon0@gmail.com" target="_blank">joe.gordon0@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi All,<div><br></div><div>A patch to move the run_tests.sh script into oslo-incubator (<a href="https://review.openstack.org/#/c/32736/" target="_blank">https://review.openstack.org/#/c/32736/</a>), has brought up the bigger question of what is the future of './run_tests.sh.'</div>
<div><br></div><div>This seems like a topic that directly affects all developers, so it sounded like it should be brought to the Mailing list.</div><div><br></div><div>Reasons to keep run_tests.sh:</div>
<div><br></div><div><div>* there is no possibility to run tests with testtools instead of testr. This feature allows us to use the debugger.</div><div>* in some projects tox doesn't use a testr wrapper to report progress while tests are running </div>
<div>* building the sdist can be slow (slow == noticeable)</div><div><br></div><div><br></div><div>Monty's Rant: <br><br>"<span style="font-family:'Arial Unicode MS',Arial,sans-serif">So, I'm SUPER torn on this.</span></div>
<p style="padding-top:0.5em;margin-bottom:0px;font-family:'Arial Unicode MS',Arial,sans-serif;margin-top:0px;padding-bottom:0.5em">On the one hand, we use tox in the gate, and run_tests.sh hides that from people and then they get confused about why some change they made to run_tests.sh doesn't get reflected in the reality of the project.</p>
<p style="padding-top:0.5em;margin-bottom:0px;font-family:'Arial Unicode MS',Arial,sans-serif;margin-top:0px;padding-bottom:0.5em">On the other hand, what Victor says is true. We need to land a couple of changes to upstream tox that would allow it to not run develop each time - but we seem to have a hard time doing that.</p>
<p style="padding-top:0.5em;margin-bottom:0px;font-family:'Arial Unicode MS',Arial,sans-serif;margin-top:0px;padding-bottom:0.5em">On the third hand, we'd doing an INSANE amount of work to make working with testing in a venv "easy" and what we've wound up with is a situation where we have multiple competing incompatible ways of doing things.</p>
<p style="padding-top:0.5em;margin-bottom:0px;font-family:'Arial Unicode MS',Arial,sans-serif;margin-top:0px;padding-bottom:0.5em">Here's what I do: source .tox/py27/bin/activate testr run --parallel testr run --parallel testr run some-test testr run --failing deactivate</p>
<p style="padding-top:0.5em;margin-bottom:0px;font-family:'Arial Unicode MS',Arial,sans-serif;margin-top:0px;padding-bottom:0.5em">Sometimes, if I'm being fancy, I'll do: virtualenv foo foo/bin/pip install -r requirements.txt -r test-requirements.txt</p>
<p style="padding-top:0.5em;margin-bottom:0px;font-family:'Arial Unicode MS',Arial,sans-serif;margin-top:0px;padding-bottom:0.5em">I believe vish just installs the depends directly on his box and skips venvs altogether.</p>
<p style="padding-top:0.5em;margin-bottom:0px;font-family:'Arial Unicode MS',Arial,sans-serif;margin-top:0px;padding-bottom:0.5em">It turns out that testr is a powerful tool with a nice UI. Instead of putting wrappers around it, perhaps we should add support for our output stream thing we like to it upstream. We should also add support to it for dropping into a debugger directly, so that the testtools issue goes away. Then we should land a patch to upstream tox to get it to do setup.py develop instead of sdist+ install.</p>
<p style="padding-top:0.5em;margin-bottom:0px;font-family:'Arial Unicode MS',Arial,sans-serif;margin-top:0px;padding-bottom:0.5em">If we did that, we could have "run tox" be the simple answer for anyone who just wants to run whatever the gate runs, because that will always work. And if they want to do fancier things, they should learn about venvs and testr.</p>
<p style="padding-top:0.5em;margin-bottom:0px;font-family:'Arial Unicode MS',Arial,sans-serif;margin-top:0px;padding-bottom:0.5em">However, if we ARE going to keep a run_tests.sh file in our trees, we should certainly have a single copy and have it behave consistently.</p>
<p style="padding-top:0.5em;margin-bottom:0px;font-family:'Arial Unicode MS',Arial,sans-serif;margin-top:0px;padding-bottom:0.5em">Please note that some projects actually do just have run_tests.sh as a thin wrapper around tox."</p>
<p style="padding-top:0.5em;margin-bottom:0px;font-family:'Arial Unicode MS',Arial,sans-serif;margin-top:0px;padding-bottom:0.5em"><br></p><p style="padding-top:0.5em;margin-bottom:0px;font-family:'Arial Unicode MS',Arial,sans-serif;margin-top:0px;padding-bottom:0.5em">
What do you think? Should we keep run_tests as is and port to oslo, or should we revisit the role of run_tests.sh?</p><p style="padding-top:0.5em;margin-bottom:0px;font-family:'Arial Unicode MS',Arial,sans-serif;margin-top:0px;padding-bottom:0.5em">
<br></p><p style="padding-top:0.5em;margin-bottom:0px;font-family:'Arial Unicode MS',Arial,sans-serif;margin-top:0px;padding-bottom:0.5em">best,</p><p style="padding-top:0.5em;margin-bottom:0px;font-family:'Arial Unicode MS',Arial,sans-serif;margin-top:0px;padding-bottom:0.5em">
Joe</p></div></div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><div class="gmail_default" style="font-family:'courier new',monospace;font-size:x-small"></div><div class="gmail_default" style="font-family:'courier new',monospace;font-size:x-small">
As far as the question of whether run_tests.sh lives or dies, I have no intention of killing it (or letting it die in Cinder).</div><div class="gmail_default" style="font-family:'courier new',monospace;font-size:x-small">
<br></div><div class="gmail_default" style="font-family:'courier new',monospace;font-size:x-small">With respect to porting it to OSLO, I'm torn a bit here. There are some things I like WRT the direction of testing in other projects, and some things not so much. The ability to have run_tests in the project allows me to tweak things and keep it usable (granted this capability probably wouldn't go away).</div>
<div class="gmail_default" style="font-family:'courier new',monospace;font-size:x-small"><br></div><div class="gmail_default" style="font-family:'courier new',monospace;font-size:x-small">I'm just a bit uneasy with all of the testing "enhancements and improvements". We've been trying to pull said changes in to Cinder but to be honest most of it from a dev perspective has been far more trouble than it's been worth. One change we loose ability to run individual tests, get it back... loose ability to run debugger... get it back... loose color output (I know horror or horrors), and so on. Understood if it was common or removed we'd all be in the same boat, but I'm not sure other care about some things like no virtual env and debugging tests as much as I do.</div>
<div class="gmail_default" style="font-family:'courier new',monospace;font-size:x-small"><br></div><div class="gmail_default" style="font-family:'courier new',monospace;font-size:x-small">Anyway, run_tests is extremely valuable to me and I believe others as is the ability to run outside of venv etc. Granted there is a ton of overlap and OSLO does have meritt, I just don't want unwanted and unnecessary change as we've finally got things stabilized again.</div>
<div class="gmail_default" style="font-family:'courier new',monospace;font-size:x-small"><br></div><div class="gmail_default" style="font-family:'courier new',monospace;font-size:x-small">Thanks,</div><div class="gmail_default" style="font-family:'courier new',monospace;font-size:x-small">
John</div><br></div></div>