Open Stack

I am good with that.
But Martin should respond since it will impact Refstack.

On Fri, Sep 10, 2021 at 10:46 AM Michael Johnson <johnsomor at gmail.com>
wrote:

> Personally I think it is cleaner to just unroll these from DDT into
> individual stub tests. There are not a lot of them and they are
> important for the interop testing, so should be cleanly tracked.
> Frankly we have spent more time talking about the problems than the
> change would take to write. grin
>
> If the interop team agrees, I am happy to do the work on the tests.
>
> Michael
>
> On Thu, Sep 9, 2021 at 10:07 AM Goutham Pacha Ravi
> <gouthampravi at gmail.com> wrote:
> >
> >
> >
> > On Thu, Sep 9, 2021 at 12:57 AM Martin Kopec <mkopec at redhat.com> wrote:
> >>
> >> From interop perspective it's also better not to have multiple tests
> with the same id.
> >> We encountered one more problem with ddt - the test names seem not to
> be generated consistently, see this:
> >> https://paste.opendev.org/show/809187/
> >> The test can have either _00009_TXT suffix or _9_TXT one.
> >>
> >> Until we figure this out, I think we will need to flag the test in
> interop - so that a skip of the test (because of the name mismatch in this
> case) won't make the whole guideline fail.
> >>
> >> Luigi's idea is great. Every test should be identified by a unique id
> and it shouldn't matter that the test is generated (ddt). Different input
> data -> different test -> different name -> different id.
> >> Let's try to explore whether having a unique id per ddt entry is
> possible.
> >
> >
> >
> > I think we could annotate the test inputs and include a unique UUID
> included in the test name/doc per test input. Something along these lines:
> https://review.opendev.org/c/openstack/manila-tempest-plugin/+/808114
> > However, adding the UUID as a test attr (a functionality of "testtools"
> [1]) will require a hook within ddt likely.
> >
> > However, as we've discovered, there's no way to avoid adding an ordinal
> into the generated test name when using DDT, and so we'll have to either
> set a PYTHONHASHSEED and disable hash randomization, or ensure that values
> appear in the same order all the time (example [2])
> >
> > [1]
> https://github.com/testing-cabal/testtools/blob/f38af6765d4e412b7a7ed1c77ddc9d68320330aa/testtools/testcase.py#L892
> > [2]
> https://review.opendev.org/c/openstack/manila-tempest-plugin/+/755859
> >
> >
> >
> >>
> >>
> >>
> >> On Wed, 8 Sept 2021 at 18:15, Luigi Toscano <ltoscano at redhat.com>
> wrote:
> >>>
> >>> On Wednesday, 8 September 2021 17:48:30 CEST Michael Johnson wrote:
> >>> > If the use of "ddt" is a big problem for the compliance testing, we
> >>> > can consider breaking these out into individual tests (likely to be
> >>> > duplicates to the existing "ddt" tests to maintain the legacy test
> >>> > UUIDs).
> >>>
> >>> I was wondering: wouldn't it be possible to expand or use ddt somehow
> to
> >>> inject different UUIDs for each generated test? You know in advance
> how many
> >>> tests are going to be generated.
> >>>
> >>> --
> >>> Luigi
> >>>
> >>>
> >>
> >>
> >> --
> >> Martin
>
>

-- 
Arkady Kanevsky, Ph.D.
Phone: 972 707-6456
Corporate Phone: 919 729-5744 ext. 8176456
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20210910/d680facb/attachment.html>