[designate][interop] test_create_all_recordset_types ddt issue
akanevsk at redhat.com
Fri Sep 10 15:53:34 UTC 2021
I am good with that.
But Martin should respond since it will impact Refstack.
On Fri, Sep 10, 2021 at 10:46 AM Michael Johnson <johnsomor at gmail.com>
> Personally I think it is cleaner to just unroll these from DDT into
> individual stub tests. There are not a lot of them and they are
> important for the interop testing, so should be cleanly tracked.
> Frankly we have spent more time talking about the problems than the
> change would take to write. grin
> If the interop team agrees, I am happy to do the work on the tests.
> On Thu, Sep 9, 2021 at 10:07 AM Goutham Pacha Ravi
> <gouthampravi at gmail.com> wrote:
> > On Thu, Sep 9, 2021 at 12:57 AM Martin Kopec <mkopec at redhat.com> wrote:
> >> From interop perspective it's also better not to have multiple tests
> with the same id.
> >> We encountered one more problem with ddt - the test names seem not to
> be generated consistently, see this:
> >> https://paste.opendev.org/show/809187/
> >> The test can have either _00009_TXT suffix or _9_TXT one.
> >> Until we figure this out, I think we will need to flag the test in
> interop - so that a skip of the test (because of the name mismatch in this
> case) won't make the whole guideline fail.
> >> Luigi's idea is great. Every test should be identified by a unique id
> and it shouldn't matter that the test is generated (ddt). Different input
> data -> different test -> different name -> different id.
> >> Let's try to explore whether having a unique id per ddt entry is
> > I think we could annotate the test inputs and include a unique UUID
> included in the test name/doc per test input. Something along these lines:
> > However, adding the UUID as a test attr (a functionality of "testtools"
> ) will require a hook within ddt likely.
> > However, as we've discovered, there's no way to avoid adding an ordinal
> into the generated test name when using DDT, and so we'll have to either
> set a PYTHONHASHSEED and disable hash randomization, or ensure that values
> appear in the same order all the time (example )
> > 
> > 
> >> On Wed, 8 Sept 2021 at 18:15, Luigi Toscano <ltoscano at redhat.com>
> >>> On Wednesday, 8 September 2021 17:48:30 CEST Michael Johnson wrote:
> >>> > If the use of "ddt" is a big problem for the compliance testing, we
> >>> > can consider breaking these out into individual tests (likely to be
> >>> > duplicates to the existing "ddt" tests to maintain the legacy test
> >>> > UUIDs).
> >>> I was wondering: wouldn't it be possible to expand or use ddt somehow
> >>> inject different UUIDs for each generated test? You know in advance
> how many
> >>> tests are going to be generated.
> >>> --
> >>> Luigi
> >> --
> >> Martin
Arkady Kanevsky, Ph.D.
Phone: 972 707-6456
Corporate Phone: 919 729-5744 ext. 8176456
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the openstack-discuss