From interop perspective it's also better not to have multiple tests with the same id. We encountered one more problem with ddt - the test names seem not to be generated consistently, see this: https://paste.opendev.org/show/809187/ The test can have either _00009_TXT suffix or _9_TXT one.
Until we figure this out, I think we will need to flag the test in interop - so that a skip of the test (because of the name mismatch in this case) won't make the whole guideline fail. Luigi's idea is great. Every test should be identified by a unique id and it shouldn't matter that the test is generated (ddt). Different input data -> different test -> different name -> different id. Let's try to explore whether having a unique id per ddt entry is possible. On Wed, 8 Sept 2021 at 18:15, Luigi Toscano <ltoscano@redhat.com> wrote:
On Wednesday, 8 September 2021 17:48:30 CEST Michael Johnson wrote:
If the use of "ddt" is a big problem for the compliance testing, we can consider breaking these out into individual tests (likely to be duplicates to the existing "ddt" tests to maintain the legacy test UUIDs).
I was wondering: wouldn't it be possible to expand or use ddt somehow to inject different UUIDs for each generated test? You know in advance how many tests are going to be generated.
-- Luigi
-- Martin