Hi designate team, I'm reaching out because we have encountered a problem with one of the tests [1]. We track the following name [2]: designate_tempest_plugin.tests.api.v2.test_recordset.RecordsetsTest.test_create_all_recordset_types That test name is actually incorrect due to the ddt and the real name of the test is created dynamically based on the test data passed via ddt. We have a consistency check which is supposed to verify that the test names tracked in our guidelines match the ones in Tempest/tempest plugins, however, there was a bug - we are fixing that by [3]. The question is, should interop track all the permutations of the test? See [4]. [1] https://opendev.org/openstack/designate-tempest-plugin/src/commit/da27a70ae2b39695ef6f03bbefb55afeacf1cdf3/designate_tempest_plugin/tests/api/v2/test_recordset.py#L87 [2] https://opendev.org/osf/interop/src/commit/d8eac71fa507c68e9c1f83a19dd77af8483d01b6/add-ons/guidelines/dns.2020.11.json#L116-L117 [3] https://review.opendev.org/c/osf/interop/+/806598 [4] https://review.opendev.org/c/osf/interop/+/807586 Thank you, -- Martin Kopec Senior Software Quality Engineer Red Hat EMEA -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20210908/3725e6f2/attachment.html>