Hi,
Dnia środa, 2 czerwca 2021 14:16:50 CEST Martin Kopec pisze:
> Hi Slawek,
>
> thanks for getting back to us and sharing new potential tests and
> capabilities from neutron-tempest-plugin.
> Let's first discuss tests which are in tempest directly please.
>
> We have done an analysis where we have cross checked tests we have in our
> guidelines with the ones (api and non-admin ones) present in tempest at the
> tempest checkout we currently use and here are the results:
> https://etherpad.opendev.org/p/refstack-test-analysis
> There are 110 and tempest.api.network tests which we don't have in any
> guideline yet.
> Could you please have a look at the list of the tests? Would it make sense
> to include them in a guideline? Would they extend any network capabilities
> we have in OpenStack Powered Platform program or would we need to create a
> new one(s)?
> https://opendev.org/osf/interop/src/branch/master/next.json
Sure. I took a look at that list today. I think that:
* tests from the group tempest.api.network.test_allowed_address_pair could be added to the "networks-l2-CRUD". Allowed_address_pairs is API extension, but it is supported by ML2 plugin since very long time, and should be available in all clouds which are using ML2 plugin.
* tests from tempest.api.network.test_dhcp_ipv6 can probably be included in "IPAM drivers" section as now I think all clouds should supports IPv6 :)
* tempest.api.network.test_floating_ips - those tests could be probably added to the "Core API L3 extension" section, but I'm not sure what are the guidlines for negative tests in the refstack,
* Tests from tempest.api.network.test_networks.BulkNetwork* - are similar to the other L2 CRUD tests but are testing basic bulk CRUD operations for Networks. So It could be IMO included in the "networks-l2-CRUD" section
* same for all other tests from tempest.api.network.test_networks and tempest.api.network.test_networks_negative modules
* Tests from tempest.api.network.test_ports can probably also be included in the "network-l2-CRUD" section as filtering is supported by core Neutron db modules,
* Tests from the tempest.api.network.test_routers module can probably go to the network-l3-CRUD section,
That are the tests which I think that may be included somehow in the refstack. But I'm not refstack expert so please forgive me if I included here too many of them or if some of them are not approriate to be there :)
>
> Thank you,
>
> On Mon, 24 May 2021 at 16:33, Slawek Kaplonski <skaplons@redhat.com> wrote:
> > Hi,
> >
> > Dnia poniedziałek, 26 kwietnia 2021 17:48:08 CEST Martin Kopec pisze:
> > > Hi everyone,
> > >
> > >
> > >
> > > I would like to further discuss the topics we covered with the neutron
> >
> > team
> >
> > > during
> > >
> > > the PTG [1].
> > >
> > >
> > >
> > > * adding address_group API capability
> > >
> > > It's tested by tests in neutron-tempest-plugin. First question is if
> >
> > tests
> >
> > > which are
> > >
> > > not directly in tempest can be a part of a non-add-on marketing program?
> > >
> > > It's possible to move them to tempest though, by the time we do so, could
> > >
> > > they be
> > >
> > > marked as advisory?
> > >
> > >
> > >
> > > * Shall we include QoS tempest tests since we don't know what share of
> > >
> > > vendors
> > >
> > > enable QoS? Could it be an add-on?
> > >
> > > These tests are also in neutron-tempest-plugin, I assume we're talking
> >
> > about
> >
> > > neutron_tempest_plugin.api.test_qos tests.
> > >
> > > If we want to include these tests, which program should they belong to?
> >
> > Do
> >
> > > we wanna
> > >
> > > create a new one?
> > >
> > >
> > >
> > > [1] https://etherpad.opendev.org/p/neutron-xena-ptg
> > >
> > >
> > >
> > > Thanks,
> > >
> > > --
> > >
> > > Martin Kopec
> > >
> > > Senior Software Quality Engineer
> > >
> > > Red Hat EMEA
> >
> > First of all, sorry that it took so long for me but I finally looked into
> > Neutron related tests and capabilities and I think we can possibly add few
> > things there:
> >
> > - For "networks-security-groups-CRUD" we can add "address_groups" API. It
> > is now supported by ML2 plugin [1]. In the neutron-tempest-plugin we just
> > have some scenario test [2] but we would probably need also API tests for
> > that, correct?
> >
> > - For networks-l3-CRUD we can optionally add port_forwarding API. This can
> > be added by service plugin [3] so it may not be enabled in all deployments.
> > But maybe there is some "optional feature" category in the RefStack, and if
> > so, this could be included there. Tests for that are in
> > neutron-tempest-plugin [4] and [5].
> >
> > - There are also 2 other service plugins, which I think could be included
> > as "optional feature" in the RefStack, but IMO don't fit exactly in any of
> > the existing groups. Those are QoS [6] and Trunks [7]. Tests for both are
> > in the neutron-tempest-plugin as well: Qos: [8] and [9], Trunk [10], [11]
> > and [12].
> >
> > Please let me know what do You think about it and if that would be ok and
> > if You want me to propose some patches with that or maybe You will propose
> > them.
> >
> > [1] https://review.opendev.org/c/openstack/neutron-lib/+/741784
> >
> > [2] https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/777833
> >
> > [3]
> > https://github.com/openstack/neutron/blob/master/neutron/services/portforwarding/pf_plugin.py
> >
> > [4]
> > https://github.com/openstack/neutron-tempest-plugin/blob/master/neutron_tempest_plugin/api/test_port_forwardings.py
> >
> > [5]
> > https://github.com/openstack/neutron-tempest-plugin/blob/master/neutron_tempest_plugin/api/test_port_forwarding_negative.py
> >
> > [6]
> > https://github.com/openstack/neutron/blob/master/neutron/services/qos/qos_plugin.py
> >
> > [7]
> > https://github.com/openstack/neutron/blob/master/neutron/services/trunk/plugin.py
> >
> > [8]
> > https://github.com/openstack/neutron-tempest-plugin/blob/master/neutron_tempest_plugin/api/test_qos.py
> >
> > [9]
> > https://github.com/openstack/neutron-tempest-plugin/blob/master/neutron_tempest_plugin/api/test_qos_negative.py
> >
> > [10]
> > https://github.com/openstack/neutron-tempest-plugin/blob/master/neutron_tempest_plugin/api/test_trunk.py
> >
> > [11]
> > https://github.com/openstack/neutron-tempest-plugin/blob/master/neutron_tempest_plugin/api/test_trunk_details.py
> >
> > [12]
> > https://github.com/openstack/neutron-tempest-plugin/blob/master/neutron_tempest_plugin/api/test_trunk_negative.py
> >
> > --
> >
> > Slawek Kaplonski
> >
> > Principal Software Engineer
> >
> > Red Hat
>
> --
> Martin Kopec
> Senior Software Quality Engineer
> Red Hat EMEA
--
Slawek Kaplonski
Principal Software Engineer
Red Hat