Open Stack

Joshua Harlow <harlowja at fastmail.com> wrote:

> Ihar Hrachyshka wrote:
>> In Neutron, we have a need to allocate random but unique addresses and
>> such. For that matter, we have a concept of exclusive resources:
>> https://github.com/openstack/neutron/tree/master/neutron/tests/common/exclusive_resources
>> that are relying on locks and shared file based resource allocation
>> storage. This is used in fullstack tests where all allocation is limited
>> to a single node.
>>
>> Another place where we randomize addresses is in our NeutronObject tests
>> where we just want to generate values good enough to pass
>> oslo.versionedobjects fields constraints:
>> https://github.com/openstack/neutron/blob/master/neutron/tests/unit/objects/test_base.py#L494
>> We have some hacks there to guarantee uniqueness where may be needed
>> (f.e. in case of lists of ip networks), but otherwise it’s just random
>> addresses. It’s good enough for unit tests because each test has its own
>> testing environment isolated from others.
>>
>> Ihar
>
> Ok good/interesting to know, if a test fails and someone wants to  
> reproduce it 'at home' what's the recommended way to do that if these are  
> all randomized (and the source of the failure say is in one of these  
> randomized data 'using' tests)
>
> Do you output the random number gen seed (this seems like it would work,  
> although a little PITA) so that people can use the same initially seed  
> (and then get the same random numbers as a result)?

(To clarify for those not part of the neutron dev team, ‘fullstack’ in  
neutron gate is not devstack/tempest, and does not involve other openstack  
services, or multiple nodes.)

No, we don’t dump a seed. I don’t think I saw a ‘fullstack’ failure that  
required an exact environment duplication to reproduce a test failure.

For neutron ‘fullstack’ tests, we run complete neutron services, and  
collect all their logs, per test scenario. Basically, every ‘fullstack’  
test case is a tiny openstack setup with just neutron services running,  
using some common resources like amqp bus, or ovs bridge emulating physical  
infrastructure. As long as service logs are sufficient to debug failures,  
it doesn’t differ much from any other failure in neutron f.e. in tempest.

It’s common for neutron to allocate a ‘random’ address from a subnet  
allocation pool for its network and instance ports, in which case we need  
to trace port-ids and whatnot to link related server/agent/tempest logs.  
(If you ask me why we randomize ip address allocation in neutron IPAM code,  
that’s actually for a reason, to reduce database contention when allocating  
addresses for parallel port requests).

Ihar