Hi,


Dnia środa, 2 czerwca 2021 14:16:50 CEST Martin Kopec pisze:

> Hi Slawek,

>

> thanks for getting back to us and sharing new potential tests and

> capabilities from neutron-tempest-plugin.

> Let's first discuss tests which are in tempest directly please.

>

> We have done an analysis where we have cross checked tests we have in our

> guidelines with the ones (api and non-admin ones) present in tempest at the

> tempest checkout we currently use and here are the results:

> https://etherpad.opendev.org/p/refstack-test-analysis

> There are 110 and tempest.api.network tests which we don't have in any

> guideline yet.

> Could you please have a look at the list of the tests? Would it make sense

> to include them in a guideline? Would they extend any network capabilities

> we have in OpenStack Powered Platform program or would we need to create a

> new one(s)?

> https://opendev.org/osf/interop/src/branch/master/next.json


Sure. I took a look at that list today. I think that:


* tests from the group tempest.api.network.test_allowed_address_pair could be added to the "networks-l2-CRUD". Allowed_address_pairs is API extension, but it is supported by ML2 plugin since very long time, and should be available in all clouds which are using ML2 plugin.


* tests from tempest.api.network.test_dhcp_ipv6 can probably be included in "IPAM drivers" section as now I think all clouds should supports IPv6 :)


* tempest.api.network.test_floating_ips - those tests could be probably added to the "Core API L3 extension" section, but I'm not sure what are the guidlines for negative tests in the refstack,


* Tests from tempest.api.network.test_networks.BulkNetwork* - are similar to the other L2 CRUD tests but are testing basic bulk CRUD operations for Networks. So It could be IMO included in the "networks-l2-CRUD" section


* same for all other tests from tempest.api.network.test_networks and tempest.api.network.test_networks_negative modules


* Tests from tempest.api.network.test_ports can probably also be included in the "network-l2-CRUD" section as filtering is supported by core Neutron db modules,


* Tests from the tempest.api.network.test_routers module can probably go to the network-l3-CRUD section,


That are the tests which I think that may be included somehow in the refstack. But I'm not refstack expert so please forgive me if I included here too many of them or if some of them are not approriate to be there :)


>

> Thank you,

>

> On Mon, 24 May 2021 at 16:33, Slawek Kaplonski <skaplons@redhat.com> wrote:

> > Hi,

> >

> > Dnia poniedziałek, 26 kwietnia 2021 17:48:08 CEST Martin Kopec pisze:

> > > Hi everyone,

> > >

> > >

> > >

> > > I would like to further discuss the topics we covered with the neutron

> >

> > team

> >

> > > during

> > >

> > > the PTG [1].

> > >

> > >

> > >

> > > * adding address_group API capability

> > >

> > > It's tested by tests in neutron-tempest-plugin. First question is if

> >

> > tests

> >

> > > which are

> > >

> > > not directly in tempest can be a part of a non-add-on marketing program?

> > >

> > > It's possible to move them to tempest though, by the time we do so, could

> > >

> > > they be

> > >

> > > marked as advisory?

> > >

> > >

> > >

> > > * Shall we include QoS tempest tests since we don't know what share of

> > >

> > > vendors

> > >

> > > enable QoS? Could it be an add-on?

> > >

> > > These tests are also in neutron-tempest-plugin, I assume we're talking

> >

> > about

> >

> > > neutron_tempest_plugin.api.test_qos tests.

> > >

> > > If we want to include these tests, which program should they belong to?

> >

> > Do

> >

> > > we wanna

> > >

> > > create a new one?

> > >

> > >

> > >

> > > [1] https://etherpad.opendev.org/p/neutron-xena-ptg

> > >

> > >

> > >

> > > Thanks,

> > >

> > > --

> > >

> > > Martin Kopec

> > >

> > > Senior Software Quality Engineer

> > >

> > > Red Hat EMEA

> >

> > First of all, sorry that it took so long for me but I finally looked into

> > Neutron related tests and capabilities and I think we can possibly add few

> > things there:

> >

> > - For "networks-security-groups-CRUD" we can add "address_groups" API. It

> > is now supported by ML2 plugin [1]. In the neutron-tempest-plugin we just

> > have some scenario test [2] but we would probably need also API tests for

> > that, correct?

> >

> > - For networks-l3-CRUD we can optionally add port_forwarding API. This can

> > be added by service plugin [3] so it may not be enabled in all deployments.

> > But maybe there is some "optional feature" category in the RefStack, and if

> > so, this could be included there. Tests for that are in

> > neutron-tempest-plugin [4] and [5].

> >

> > - There are also 2 other service plugins, which I think could be included

> > as "optional feature" in the RefStack, but IMO don't fit exactly in any of

> > the existing groups. Those are QoS [6] and Trunks [7]. Tests for both are

> > in the neutron-tempest-plugin as well: Qos: [8] and [9], Trunk [10], [11]

> > and [12].

> >

> > Please let me know what do You think about it and if that would be ok and

> > if You want me to propose some patches with that or maybe You will propose

> > them.

> >

> > [1] https://review.opendev.org/c/openstack/neutron-lib/+/741784

> >

> > [2] https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/777833

> >

> > [3]

> > https://github.com/openstack/neutron/blob/master/neutron/services/portforwarding/pf_plugin.py

> >

> > [4]

> > https://github.com/openstack/neutron-tempest-plugin/blob/master/neutron_tempest_plugin/api/test_port_forwardings.py

> >

> > [5]

> > https://github.com/openstack/neutron-tempest-plugin/blob/master/neutron_tempest_plugin/api/test_port_forwarding_negative.py

> >

> > [6]

> > https://github.com/openstack/neutron/blob/master/neutron/services/qos/qos_plugin.py

> >

> > [7]

> > https://github.com/openstack/neutron/blob/master/neutron/services/trunk/plugin.py

> >

> > [8]

> > https://github.com/openstack/neutron-tempest-plugin/blob/master/neutron_tempest_plugin/api/test_qos.py

> >

> > [9]

> > https://github.com/openstack/neutron-tempest-plugin/blob/master/neutron_tempest_plugin/api/test_qos_negative.py

> >

> > [10]

> > https://github.com/openstack/neutron-tempest-plugin/blob/master/neutron_tempest_plugin/api/test_trunk.py

> >

> > [11]

> > https://github.com/openstack/neutron-tempest-plugin/blob/master/neutron_tempest_plugin/api/test_trunk_details.py

> >

> > [12]

> > https://github.com/openstack/neutron-tempest-plugin/blob/master/neutron_tempest_plugin/api/test_trunk_negative.py

> >

> > --

> >

> > Slawek Kaplonski

> >

> > Principal Software Engineer

> >

> > Red Hat

>

> --

> Martin Kopec

> Senior Software Quality Engineer

> Red Hat EMEA



--

Slawek Kaplonski

Principal Software Engineer

Red Hat