Open Stack

Hi,

The tests below are failing in the tempest API / Scenario job that
runs in the networking-ovn gate (non-voting):

neutron_tempest_plugin.api.admin.test_quotas_negative.QuotasAdminNegativeTestJSON.test_create_port_when_quotas_is_full
neutron_tempest_plugin.api.test_routers.RoutersIpV6Test.test_router_interface_status
neutron_tempest_plugin.api.test_routers.RoutersTest.test_router_interface_status
neutron_tempest_plugin.api.test_subnetpools.SubnetPoolsTest.test_create_subnet_from_pool_with_prefixlen
neutron_tempest_plugin.api.test_subnetpools.SubnetPoolsTest.test_create_subnet_from_pool_with_quota
neutron_tempest_plugin.api.test_subnetpools.SubnetPoolsTest.test_create_subnet_from_pool_with_subnet_cidr

Digging a bit into it I noticed that with the exception of the two
"test_router_interface_status" (ipv6 and ipv4) all other tests are
failing because the way metadata works in networking-ovn.

Taking the "test_create_port_when_quotas_is_full" as an example. The
reason why it fails is because when the OVN metadata is enabled,
networking-ovn will metadata port at the moment a network is created
[0] and that will already fulfill the quota limit set by that test
[1].

That port will also allocate an IP from the subnet which will cause
the rest of the tests to fail with a "No more IP addresses available
on network ..." error.

This is not very trivial to fix because:

1. Tempest should be backend agnostic. So, adding a conditional in the
tempest test to check whether OVN is being used or not doesn't sound
correct.

2. Creating a port to be used by the metadata agent is a core part of
the design implementation for the metadata functionality [2]

So, I'm sending this email to try to figure out what would be the best
approach to deal with this problem and start working towards having
that job to be voting in our gate. Here are some ideas:

1. Simple disable the tests that are affected by the metadata approach.

2. Disable metadata for the tempest API / Scenario tests (here's a
test patch doing it [3])

3. Same as 1. but also create similar tempest tests specific for OVN
somewhere else (in the networking-ovn tree?!)

What you think would be the best way to workaround this problem, any
other ideas ?

As for the "test_router_interface_status" tests that are failing
independent of the metadata, there's a bug reporting the problem here
[4]. So we should just fix it.

[0] https://github.com/openstack/networking-ovn/blob/f3f5257fc465bbf44d589cc16e9ef7781f6b5b1d/networking_ovn/common/ovn_client.py#L1154
[1] https://github.com/openstack/neutron-tempest-plugin/blob/35bf37d1830328d72606f9c790b270d4fda2b854/neutron_tempest_plugin/api/admin/test_quotas_negative.py#L66
[2] https://docs.openstack.org/networking-ovn/latest/contributor/design/metadata_api.html#overview-of-proposed-approach
[3] https://review.openstack.org/#/c/558792/
[4] https://bugs.launchpad.net/networking-ovn/+bug/1713835

Cheers,
Lucas