[openstack-dev] [qa][Neutron][Tempest][Network] Break down NetworkBasicOps to smaller test cases

David Kranz dkranz at redhat.com
Wed Jan 22 20:40:29 UTC 2014


On 01/22/2014 03:19 PM, Jay Pipes wrote:
> On Tue, 2014-01-21 at 01:15 -0500, Yair Fried wrote:
>> I seem to be unable to convey my point using generalization, so I will give a specific example:
>> I would like to have "update dns server" as an additional network scenario. Currently I could add it to the existing module:
>>
>> 1. tests connectivity
>> 2. re-associate floating ip
>> 3. update dns server
>>
>> In which case, failure to re-associate ip will prevent my test from running, even though these are completely unrelated scenarios, and (IMO) we would like to get feedback on both of them.
>>
>> Another way, is to copy the entire network_basic_ops module, remove "re-associate floating ip" and add "update dns server". For the obvious reasons - this also seems like the wrong way to go.
>>
>> I am looking for an elegant way to share the code of these scenarios.
> Well, unfortunately, there are no very elegant answers at this time :)
>
> The closest thing we have would be to create a fixtures.Fixture that
> constructed a VM and associated the floating IP address to the instance.
> You could then have separate tests that for checking connectivity and
> updating the DNS server for that instance. However, fixtures are for
> resources that are shared between test methods and are not modified
> during those test methods. They cannot be modified, because then
> parallel execution of the test methods may yield non-deterministic
> results.
>
> There would need to be a separate fixture for the instance that would
> have the floating IP re-associated with it (because re-associating the
> floating IP by nature is a modification to the resource).
>
> Having a separate fixture means essentially doubling the amount of
> resources used by the test case class in question, which is why we're
> pushing to just have all of the tests done serially in a single test
> method, even though that means that a failure to re-associate the
> floating IP would mean that the update DNS server test would not be
> executed.
>
> Choose your poison.
>
> Best,
> -jay
>
>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Thanks, Jay. So to close this loop, I think Yair started down this road 
after receiving feedback that this test was getting too much stuff in 
it. Sounds like you are advocating putting more stuff in it as the least 
of evils. Which is fine by me because it is a lot easier.

  -David



More information about the OpenStack-dev mailing list