[openstack-dev] [qa][Neutron][Tempest][Network] Break down NetworkBasicOps to smaller test cases

Salvatore Orlando sorlando at nicira.com
Mon Jan 20 17:22:22 UTC 2014


Yair is probably referring to statistically independent tests, or whatever
case for which the following is true (P(x) is the probably that a test
succeeds):

P(4|3|2|1) = P(4|1) * P(3|1) * P(2|1)

This might apply to the tests we are adding to network_basic_ops scenario;
however it is worth noting that:

- in some cases the above relationship does not hold. For instance a public
network connectivity test can hardly succeeds if the private connectivity
test failed (is that correct? I'm not sure anymore of anything this days!)
- Sean correctly pointed out that splitting test will cause repeated
activities which will just make the test run longer without any additional
benefit.

On the other hand, I understand and share the feeling that we are adding
too much to the same workflow. Would it make sense to identify a few
conceptually independent workflows, identify one or more advanced network
scenarios, and keep only internal + public connectivity checks in basic_ops?

Salvatore


On 20 January 2014 09:23, Jay Pipes <jaypipes at gmail.com> wrote:

> On Sun, 2014-01-19 at 07:17 -0500, Yair Fried wrote:
> > OK,
> > but considering my pending patch (#3 and #4)
> > what about:
> >
> > #1 -> #2
> > #1 -> #3
> > #1 -> #4
> >
> > instead of
> >
> > #1 -> #2 -> #3 -> #4
> >
> > a failure in #2 will prevent #3 and #4 from running even though they are
> completely unrelated
>
> Seems to me, that the above is a logical fault. If a failure in #2
> prevents #3 or #4 from running, then by nature they are related to #2.
>
> -jay
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140120/612efc8d/attachment.html>


More information about the OpenStack-dev mailing list