[openstack-dev] [QA] [Ironic] [Inspector] Where should integration tests for non-core projects live now? (Was: Toward 2.0.0 release)
Yuiko Takada
yuikotakada0313 at gmail.com
Wed Jun 17 03:38:59 UTC 2015
Hi,
Dmitry, thank you for putting it up, and also
Ken'ichi, thank you for your reply.
> Will it be possible to run tests on Ironic as well using plugin from
> > ironic-inspector?
>
> Yeah, it will be possible.
> but I'm guessing ironic-inspector is optional and Ironic should not
> depend on the gate test result of ironic-inspector.
> So maybe you just need to run Ironic tests on ironic-inspector gate
> tests, right?
Exactly. All we want to do is run Ironic+Ironic-inspector tests on gate.
Then, as you and Matt and Dimitry talked about this on IRC few days ago,
We can add Ironic/Ironic-inspector tests into Tempest still, right?
So that I've started to implement a test in Tempest,
but I'm facing another issue.
As you know, Ironic API has microversions, and Ironic-inspector can run
with microversion > 1.6.
But currently there is no feature testing specific Ironic API microversions
on Tempest, right?
So that, we have to think about some solutions.
(1) Make testing specific Ironic API microversions on Tempest possible
adam_g is posting this patch set.
https://review.openstack.org/166386
(2)Using tempest_lib instead of adding tests into Tempest
Is tempest_lib available already?
Or do we need to wait for something will be merged?
(3)Make Ironic-inspector available even if microversion < 1.6
Dmitry is posting this patch set.
https://review.openstack.org/192196
# I don't mean asking you to review this, don't worry :p
Could you please think about the best and fast solution together?
Best Regards,
Yuiko Takada
2015-06-10 17:57 GMT+09:00 Ken'ichi Ohmichi <ken1ohmichi at gmail.com>:
> 2015-06-10 16:48 GMT+09:00 Dmitry Tantsur <dtantsur at redhat.com>:
> > On 06/10/2015 09:40 AM, Ken'ichi Ohmichi wrote:
> >> To solve it, we have decided the scope of Tempest as the etherpad
> >> mentioned.
> >>
> >>> Are there any hints now on where we can start with our integration
> tests?
> >>
> >>
> >> For the other projects, we are migrating the test framework of Tempest
> >> to tempest-lib which is a library.
> >> So each project can implement their own tests in each repository by
> >> using the test framework of tempest-lib.
> >
> >
> > So in my case we can start with putting test code to ironic-inspector
> tree
> > using tempest-lib, right?
>
> Yeah, right.
> Neutron is already doing that.
> maybe neutron/tests/api/ of Neutron repository will be a hint for it.
>
> > Will it be possible to run tests on Ironic as well using plugin from
> > ironic-inspector?
>
> Yeah, it will be possible.
> but I'm guessing ironic-inspector is optional and Ironic should not
> depend on the gate test result of ironic-inspector.
> So maybe you just need to run Ironic tests on ironic-inspector gate
> tests, right?
>
> >>> After a quick look at devstack-gate I got an impression that it's
> >>> expecting
> >>> tests as part of tempest:
> >>>
> >>>
> https://github.com/openstack-infra/devstack-gate/blob/master/devstack-vm-gate.sh#L600
> >>>
> >>> Our final goal is to have devstack gate test for Ironic and Inspector
> >>> projects working together.
> >>
> >>
> >> We have discussed external interfaces of Tempest on the summit, so
> >> that Tempest gathers tests from each project repository and runs them
> >> at the same time.
> >> There is a qa-spec for https://review.openstack.org/#/c/184992/
> >
> >
> > Cool, thanks! Does it mean that devstack-gate will also be updated to
> allow
> > something like DEVSTACK_GATE_TEMPEST_PLUGINS="https://github.com/..."?
>
> Yeah, will be.
> The idea of this external interface is based on DevStack's one.
> I think we will be able to use it on the gate like that.
>
> Thanks
> Ken'ichi Ohmichi
>
> ---
>
> >>> On 06/10/2015 08:07 AM, Yuiko Takada wrote:
> >>>>
> >>>>
> >>>> Hi, Dmitry,
> >>>>
> >>>> I guess the whole idea of new release models is NOT to tie
> projects
> >>>> to each other any more except for The Big Release twice a year :)
> >>>> So
> >>>> I think no, we don't need to. We still can do it, if we have
> >>>> something to release by the time Ironic releases, but I suggest
> >>>> deciding it on case-by-case basis.
> >>>>
> >>>> OK, I see.
> >>>>
> >>>> One more concern, about Tempest integration test which I will
> implement
> >>>> in V2.1.0,
> >>>> it seems like that we cannot add Ironic-inspector's tests into Tempest
> >>>> even if integration tests.
> >>>> Please see:
> >>>> https://etherpad.openstack.org/p/YVR-QA-in-the-big-tent
> >>>
> >>>
> >>>
> >>> Good catch. I guess the answer depends on where Ironic integration
> tests
> >>> are
> >>> going to live - we're going to live with them. Let me retarget this
> >>> thread
> >>> to a wider audience.
> >>>
> >>>>
> >>>> But I heard from you that Devananda thinks we need this in tempest
> >>>> itself. [3]
> >>>> Do you know something like current situation?
> >>>>
> >>>>
> >>>> Best Regards,
> >>>> Yuiko Takada
> >>>>
> >>>> 2015-06-09 15:59 GMT+09:00 Dmitry Tantsur <dtantsur at redhat.com
> >>>> <mailto:dtantsur at redhat.com>>:
> >>>>
> >>>> On 06/09/2015 03:49 AM, Yuiko Takada wrote:
> >>>>
> >>>> Hi, Dmitry,
> >>>>
> >>>> Thank you for notifying.
> >>>>
> >>>> I've updated our summit etherpad [3] with whatever
> >>>> priorities
> >>>> I
> >>>> remembered, please have a look. I've also untargeted a
> few
> >>>> things in
> >>>> launchpad [4] (and will probably untarget more later
> on).
> >>>> Please
> >>>> assign yourself, if you want something done in this
> >>>> release
> >>>> time frame.
> >>>>
> >>>> I've assigned one item to myself in [3], and also I added one
> >>>> BP
> >>>> to [4],
> >>>> so please take a look.
> >>>>
> >>>> https://blueprints.launchpad.net/ironic-inspector/+spec/delete-db-api
> >>>>
> >>>>
> >>>> Looks good, though I don't think it's a big priority for 2.0.0.
> >>>> Definitely worth doing for 2.1.0.
> >>>>
> >>>> Thanks for assigning for tempest work, that's definitely a
> priority
> >>>> right now.
> >>>>
> >>>>
> >>>> BTW, how do you think about Ironic-inspector's release model?
> >>>> You wrote "Version released with Ironic Liberty" as
> >>>> Ironic-inspector Version 2.1.0 in etherpad [3],
> >>>> but as you know, Ironic's release model has changed to
> feature
> >>>> releases.(right?)
> >>>> Should we make our release model same as Ironic?
> >>>>
> >>>>
> >>>> I guess the whole idea of new release models is NOT to tie
> projects
> >>>> to each other any more except for The Big Release twice a year :)
> >>>> So
> >>>> I think no, we don't need to. We still can do it, if we have
> >>>> something to release by the time Ironic releases, but I suggest
> >>>> deciding it on case-by-case basis.
> >>>>
> >>>>
> >>>>
> >>>> Best Regards,
> >>>> Yuiko Takada(Inspector team member)
> >>>>
> >>>> 2015-06-08 20:38 GMT+09:00 Dmitry Tantsur <
> dtantsur at redhat.com
> >>>> <mailto:dtantsur at redhat.com>
> >>>> <mailto:dtantsur at redhat.com <mailto:dtantsur at redhat.com>>>:
> >>>>
> >>>>
> >>>> Hello, Inspector team!
> >>>>
> >>>> The renaming process is going pretty well, the last
> thing
> >>>> we need to
> >>>> do is to get Infra approval and actual rename [1][2].
> >>>>
> >>>> I'd like to allow people (e.g. myself) to start
> packaging
> >>>> inspector
> >>>> under it's new name, so I'd like to make 2.0.0 release
> as
> >>>> soon as
> >>>> possible (as opposed to scheduling it to particular
> date).
> >>>> All
> >>>> breaking changes should land by this release - I don't
> >>>> expect 3.0.0
> >>>> soon :)
> >>>>
> >>>> I've updated our summit etherpad [3] with whatever
> >>>> priorities
> >>>> I
> >>>> remembered, please have a look. I've also untargeted a
> few
> >>>> things in
> >>>> launchpad [4] (and will probably untarget more later
> on).
> >>>> Please
> >>>> assign yourself, if you want something done in this
> >>>> release
> >>>> time frame.
> >>>>
> >>>> I would like 2.1.0 to be released with Ironic Liberty
> and
> >>>> be
> >>>> properly supported.
> >>>>
> >>>> Let me know what you think.
> >>>>
> >>>> Cheers,
> >>>> Dmitry
> >>>>
> >>>> [1] https://review.openstack.org/#/c/188030/
> >>>> [2] https://review.openstack.org/#/c/188798/
> >>>> [3]
> >>>> https://etherpad.openstack.org/p/liberty-ironic-discoverd
> >>>> [4]
> >>>> https://bugs.launchpad.net/ironic-inspector/+milestone/2.0.0
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> __________________________________________________________________________
> >>>> OpenStack Development Mailing List (not for usage
> >>>> questions)
> >>>> Unsubscribe:
> >>>>
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >>>>
> >>>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
> >
> >>>>
> >>>>
> >>>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
> >
> >>>>
> >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> __________________________________________________________________________
> >>>> OpenStack Development Mailing List (not for usage questions)
> >>>> Unsubscribe:
> >>>>
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >>>>
> >>>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
> >
> >>>>
> >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> __________________________________________________________________________
> >>>> OpenStack Development Mailing List (not for usage questions)
> >>>> Unsubscribe:
> >>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >>>>
> >>>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
> >
> >>>>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> __________________________________________________________________________
> >>>> OpenStack Development Mailing List (not for usage questions)
> >>>> Unsubscribe:
> >>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>>
> >>>
> >>>
> >>>
> >>>
> __________________________________________________________________________
> >>> OpenStack Development Mailing List (not for usage questions)
> >>> Unsubscribe:
> >>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >>
> >>
> __________________________________________________________________________
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >
> >
> >
> __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150617/e3dc8901/attachment.html>
More information about the OpenStack-dev
mailing list