<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<div dir="ltr">Another option is to just run separate test jobs. Like Django tests in one runner, and angular in another. As different panels are deprecated and removed, and we alter the scope of the tests, two jobs might act as a good stopgap.
<div><br>
</div>
<div>Rob</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On 21 July 2016 at 01:58, Richard Jones <span dir="ltr">
<<a href="mailto:r1chardj0n3s@gmail.com" target="_blank">r1chardj0n3s@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote"><span class="">On 21 July 2016 at 10:13, Timur Sufiev <span dir="ltr">
<<a href="mailto:tsufiev@mirantis.com" target="_blank">tsufiev@mirantis.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">I'm not sure if (3) was the thing we agreed upon (and if it will help us to detect issues in Horizon) but the rest of options definitely make sense to me.</div>
</blockquote>
<div><br>
</div>
</span>
<div>I think that's reasonable - moving the tests to tempest will make it more difficult to run them locally.</div>
<span class="">
<div><br>
</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div>Also there is <br>
</div>
<div><br>
</div>
<div>5. Parallelize integration tests.</div>
</div>
</blockquote>
<div><br>
</div>
</span>
<div>Ah, yep, good catch. I'm not sure whether this will negatively impact our stability on some of the test nodes though :/</div>
<span class="HOEnZb"><font color="#888888">
<div><br>
</div>
<div><br>
</div>
<div> Richard</div>
</font></span><span class="">
<div><br>
</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="gmail_quote">
<div>
<div>
<div dir="ltr">On Wed, Jul 20, 2016 at 4:28 PM Richard Jones <<a href="mailto:r1chardj0n3s@gmail.com" target="_blank">r1chardj0n3s@gmail.com</a>> wrote:<br>
</div>
</div>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div>
<div dir="ltr">Hi folks,
<div><br>
</div>
<div>Sorry I didn't make the meeting this morning :-(</div>
<div><br>
</div>
<div>We touched on testing at the mid-cycle and again it came up in the meeting this morning. We identified two issues:</div>
<div><br>
</div>
<div>1. Our integration test suite cannot grow to too many more tests because they take too long to run (and get auto-killed). We could increase the amount of time allowed for them, but we agreed that we shouldn't do this.</div>
<div>2. We don't have enough higher-level (integration leve) test coverage for our newer angular interfaces.</div>
<div><br>
</div>
<div>These two are obviously at odds :-)</div>
<div><br>
</div>
<div>What we agreed on, I think, is that we're going to:</div>
<div><br>
</div>
<div>1. Reduce the granularity of the integration tests - use them to broadly test whether an interface works at all when presented to a browser,</div>
<div>2. Replace many single interface tests with a fewer tests that poke at many interfaces (thus removing the significant per-test overhead) - even potentially having a single test that attempts to access every panel (as a guard against gross Javascript incompatibilities
or configuration issues breaking panels),</div>
<div>3. Move the integration tests to tempest, and</div>
<div>4. Write some "django unit test suite" functional tests for our angular code that tests more than just the single units we currently test (ie. mock at a more remote level, checking that broader interactions work).</div>
<div><br>
</div>
<div>Does this sound about right?</div>
<div><br>
</div>
<div><br>
</div>
<div> Richard</div>
<div><br>
</div>
</div>
</div>
</div>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</blockquote>
</div>
<br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br>
</blockquote>
</span></div>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</body>
</html>