[openstack-dev] [Horizon] On testing...

Rob Cresswell robert.cresswell at outlook.com
Thu Jul 21 08:38:03 UTC 2016


Another option is to just run separate test jobs. Like Django tests in one runner, and angular in another. As different panels are deprecated and removed, and we alter the scope of the tests, two jobs might act as a good stopgap.

Rob

On 21 July 2016 at 01:58, Richard Jones <r1chardj0n3s at gmail.com<mailto:r1chardj0n3s at gmail.com>> wrote:
On 21 July 2016 at 10:13, Timur Sufiev <tsufiev at mirantis.com<mailto:tsufiev at mirantis.com>> wrote:
I'm not sure if (3) was the thing we agreed upon (and if it will help us to detect issues in Horizon) but the rest of options definitely make sense to me.

I think that's reasonable - moving the tests to tempest will make it more difficult to run them locally.


Also there is

5. Parallelize integration tests.

Ah, yep, good catch. I'm not sure whether this will negatively impact our stability on some of the test nodes though :/


     Richard


On Wed, Jul 20, 2016 at 4:28 PM Richard Jones <r1chardj0n3s at gmail.com<mailto:r1chardj0n3s at gmail.com>> wrote:
Hi folks,

Sorry I didn't make the meeting this morning :-(

We touched on testing at the mid-cycle and again it came up in the meeting this morning. We identified two issues:

1. Our integration test suite cannot grow to too many more tests because they take too long to run (and get auto-killed). We could increase the amount of time allowed for them, but we agreed that we shouldn't do this.
2. We don't have enough higher-level (integration leve) test coverage for our newer angular interfaces.

These two are obviously at odds :-)

What we agreed on, I think, is that we're going to:

1. Reduce the granularity of the integration tests - use them to broadly test whether an interface works at all when presented to a browser,
2. Replace many single interface tests with a fewer tests that poke at many interfaces (thus removing the significant per-test overhead) - even potentially having a single test that attempts to access every panel (as a guard against gross Javascript incompatibilities or configuration issues breaking panels),
3. Move the integration tests to tempest, and
4. Write some "django unit test suite" functional tests for our angular code that tests more than just the single units we currently test (ie. mock at a more remote level, checking that broader interactions work).

Does this sound about right?


     Richard

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160721/c90a7c04/attachment.html>


More information about the OpenStack-dev mailing list