Open Stack

I seem to be unable to convey my point using generalization, so I will give a specific example:
I would like to have "update dns server" as an additional network scenario. Currently I could add it to the existing module:

1. tests connectivity
2. re-associate floating ip
3. update dns server

In which case, failure to re-associate ip will prevent my test from running, even though these are completely unrelated scenarios, and (IMO) we would like to get feedback on both of them.

Another way, is to copy the entire network_basic_ops module, remove "re-associate floating ip" and add "update dns server". For the obvious reasons - this also seems like the wrong way to go.

I am looking for an elegant way to share the code of these scenarios.

Yair


----- Original Message -----
From: "Salvatore Orlando" <sorlando at nicira.com>
To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org>
Sent: Monday, January 20, 2014 7:22:22 PM
Subject: Re: [openstack-dev] [qa][Neutron][Tempest][Network] Break down NetworkBasicOps to smaller test cases



Yair is probably referring to statistically independent tests, or whatever case for which the following is true (P(x) is the probably that a test succeeds): 


P(4|3|2|1) = P(4|1) * P(3|1) * P(2|1) 



This might apply to the tests we are adding to network_basic_ops scenario; however it is worth noting that: 


- in some cases the above relationship does not hold. For instance a public network connectivity test can hardly succeeds if the private connectivity test failed (is that correct? I'm not sure anymore of anything this days!) 
- Sean correctly pointed out that splitting test will cause repeated activities which will just make the test run longer without any additional benefit. 


On the other hand, I understand and share the feeling that we are adding too much to the same workflow. Would it make sense to identify a few conceptually independent workflows, identify one or more advanced network scenarios, and keep only internal + public connectivity checks in basic_ops? 


Salvatore 



On 20 January 2014 09:23, Jay Pipes < jaypipes at gmail.com > wrote: 



On Sun, 2014-01-19 at 07:17 -0500, Yair Fried wrote: 
> OK, 
> but considering my pending patch (#3 and #4) 
> what about: 
> 
> #1 -> #2 
> #1 -> #3 
> #1 -> #4 
> 
> instead of 
> 
> #1 -> #2 -> #3 -> #4 
> 
> a failure in #2 will prevent #3 and #4 from running even though they are completely unrelated 

Seems to me, that the above is a logical fault. If a failure in #2 
prevents #3 or #4 from running, then by nature they are related to #2. 

-jay 




_______________________________________________ 
OpenStack-dev mailing list 
OpenStack-dev at lists.openstack.org 
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev 


_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev