[openstack-dev] [QA] [Ironic] [Inspector] Where should integration tests for non-core projects live now? (Was: Toward 2.0.0 release)
Dmitry Tantsur
dtantsur at redhat.com
Wed Jun 10 10:23:45 UTC 2015
On 06/10/2015 11:57 AM, Boris Pavlovic wrote:
> Dmitry,
>
> If you chose to use Rally framework for testing there are 3 opportunities:
>
> - Keep Rally plugins (tests) in separated tree
> - Keep Rally plugins (tests) in your project tree
> - Keep Rally plugins (tests) in Rally repo
>
> Rally plugins can be used for all kinds of testing: (perf, scalability,
> load...)
> so you are killing two birds with one stone.
>
> P.S. I would imho prefer to keep all high quality plugins inside Rally
> repo to simplify operators life..
Hi, that sounds interesting, I'll have a look.
Note, however, that Inspector integration testing highly depends on
Ironic one, so unless Ironic adapts/agrees to adapt Rally, it will be
hard to Inspector to do it.
>
>
> Best regards,
> Boris Pavlovic
>
> On Wed, Jun 10, 2015 at 11:57 AM, Ken'ichi Ohmichi
> <ken1ohmichi at gmail.com <mailto:ken1ohmichi at gmail.com>> wrote:
>
> 2015-06-10 16:48 GMT+09:00 Dmitry Tantsur <dtantsur at redhat.com
> <mailto:dtantsur at redhat.com>>:
> > On 06/10/2015 09:40 AM, Ken'ichi Ohmichi wrote:
> >> To solve it, we have decided the scope of Tempest as the etherpad
> >> mentioned.
> >>
> >>> Are there any hints now on where we can start with our integration tests?
> >>
> >>
> >> For the other projects, we are migrating the test framework of Tempest
> >> to tempest-lib which is a library.
> >> So each project can implement their own tests in each repository by
> >> using the test framework of tempest-lib.
> >
> >
> > So in my case we can start with putting test code to ironic-inspector tree
> > using tempest-lib, right?
>
> Yeah, right.
> Neutron is already doing that.
> maybe neutron/tests/api/ of Neutron repository will be a hint for it.
>
> > Will it be possible to run tests on Ironic as well using plugin from
> > ironic-inspector?
>
> Yeah, it will be possible.
> but I'm guessing ironic-inspector is optional and Ironic should not
> depend on the gate test result of ironic-inspector.
> So maybe you just need to run Ironic tests on ironic-inspector gate
> tests, right?
>
> >>> After a quick look at devstack-gate I got an impression that it's
> >>> expecting
> >>> tests as part of tempest:
> >>>
> >>>https://github.com/openstack-infra/devstack-gate/blob/master/devstack-vm-gate.sh#L600
> >>>
> >>> Our final goal is to have devstack gate test for Ironic and Inspector
> >>> projects working together.
> >>
> >>
> >> We have discussed external interfaces of Tempest on the summit, so
> >> that Tempest gathers tests from each project repository and runs them
> >> at the same time.
> >> There is a qa-spec forhttps://review.openstack.org/#/c/184992/
> >
> >
> > Cool, thanks! Does it mean that devstack-gate will also be updated to allow
> > something like DEVSTACK_GATE_TEMPEST_PLUGINS="https://github.com/..."?
>
> Yeah, will be.
> The idea of this external interface is based on DevStack's one.
> I think we will be able to use it on the gate like that.
>
> Thanks
> Ken'ichi Ohmichi
>
> ---
>
> >>> On 06/10/2015 08:07 AM, Yuiko Takada wrote:
> >>>>
> >>>>
> >>>> Hi, Dmitry,
> >>>>
> >>>> I guess the whole idea of new release models is NOT to
> tie projects
> >>>> to each other any more except for The Big Release twice a
> year :)
> >>>> So
> >>>> I think no, we don't need to. We still can do it, if we have
> >>>> something to release by the time Ironic releases, but I
> suggest
> >>>> deciding it on case-by-case basis.
> >>>>
> >>>> OK, I see.
> >>>>
> >>>> One more concern, about Tempest integration test which I will
> implement
> >>>> in V2.1.0,
> >>>> it seems like that we cannot add Ironic-inspector's tests into
> Tempest
> >>>> even if integration tests.
> >>>> Please see:
> >>>> https://etherpad.openstack.org/p/YVR-QA-in-the-big-tent
> >>>
> >>>
> >>>
> >>> Good catch. I guess the answer depends on where Ironic
> integration tests
> >>> are
> >>> going to live - we're going to live with them. Let me retarget this
> >>> thread
> >>> to a wider audience.
> >>>
> >>>>
> >>>> But I heard from you that Devananda thinks we need this in tempest
> >>>> itself. [3]
> >>>> Do you know something like current situation?
> >>>>
> >>>>
> >>>> Best Regards,
> >>>> Yuiko Takada
> >>>>
> >>>> 2015-06-09 15:59 GMT+09:00 Dmitry Tantsur <dtantsur at redhat.com
> <mailto:dtantsur at redhat.com>
> >>>> <mailto:dtantsur at redhat.com <mailto:dtantsur at redhat.com>>>:
> >>>>
> >>>> On 06/09/2015 03:49 AM, Yuiko Takada wrote:
> >>>>
> >>>> Hi, Dmitry,
> >>>>
> >>>> Thank you for notifying.
> >>>>
> >>>> I've updated our summit etherpad [3] with whatever
> >>>> priorities
> >>>> I
> >>>> remembered, please have a look. I've also
> untargeted a few
> >>>> things in
> >>>> launchpad [4] (and will probably untarget more
> later on).
> >>>> Please
> >>>> assign yourself, if you want something done in this
> >>>> release
> >>>> time frame.
> >>>>
> >>>> I've assigned one item to myself in [3], and also I
> added one
> >>>> BP
> >>>> to [4],
> >>>> so please take a look.
> >>>>
> >>>>
> https://blueprints.launchpad.net/ironic-inspector/+spec/delete-db-api
> >>>>
> >>>>
> >>>> Looks good, though I don't think it's a big priority for
> 2.0.0.
> >>>> Definitely worth doing for 2.1.0.
> >>>>
> >>>> Thanks for assigning for tempest work, that's definitely
> a priority
> >>>> right now.
> >>>>
> >>>>
> >>>> BTW, how do you think about Ironic-inspector's
> release model?
> >>>> You wrote "Version released with Ironic Liberty" as
> >>>> Ironic-inspector Version 2.1.0 in etherpad [3],
> >>>> but as you know, Ironic's release model has changed
> to feature
> >>>> releases.(right?)
> >>>> Should we make our release model same as Ironic?
> >>>>
> >>>>
> >>>> I guess the whole idea of new release models is NOT to
> tie projects
> >>>> to each other any more except for The Big Release twice a
> year :)
> >>>> So
> >>>> I think no, we don't need to. We still can do it, if we have
> >>>> something to release by the time Ironic releases, but I
> suggest
> >>>> deciding it on case-by-case basis.
> >>>>
> >>>>
> >>>>
> >>>> Best Regards,
> >>>> Yuiko Takada(Inspector team member)
> >>>>
> >>>> 2015-06-08 20:38 GMT+09:00 Dmitry Tantsur
> <dtantsur at redhat.com <mailto:dtantsur at redhat.com>
> >>>> <mailto:dtantsur at redhat.com <mailto:dtantsur at redhat.com>>
> >>>> <mailto:dtantsur at redhat.com
> <mailto:dtantsur at redhat.com> <mailto:dtantsur at redhat.com
> <mailto:dtantsur at redhat.com>>>>:
> >>>>
> >>>>
> >>>> Hello, Inspector team!
> >>>>
> >>>> The renaming process is going pretty well, the
> last thing
> >>>> we need to
> >>>> do is to get Infra approval and actual rename
> [1][2].
> >>>>
> >>>> I'd like to allow people (e.g. myself) to start
> packaging
> >>>> inspector
> >>>> under it's new name, so I'd like to make 2.0.0
> release as
> >>>> soon as
> >>>> possible (as opposed to scheduling it to
> particular date).
> >>>> All
> >>>> breaking changes should land by this release - I
> don't
> >>>> expect 3.0.0
> >>>> soon :)
> >>>>
> >>>> I've updated our summit etherpad [3] with whatever
> >>>> priorities
> >>>> I
> >>>> remembered, please have a look. I've also
> untargeted a few
> >>>> things in
> >>>> launchpad [4] (and will probably untarget more
> later on).
> >>>> Please
> >>>> assign yourself, if you want something done in this
> >>>> release
> >>>> time frame.
> >>>>
> >>>> I would like 2.1.0 to be released with Ironic
> Liberty and
> >>>> be
> >>>> properly supported.
> >>>>
> >>>> Let me know what you think.
> >>>>
> >>>> Cheers,
> >>>> Dmitry
> >>>>
> >>>> [1] https://review.openstack.org/#/c/188030/
> >>>> [2] https://review.openstack.org/#/c/188798/
> >>>> [3]
> >>>> https://etherpad.openstack.org/p/liberty-ironic-discoverd
> >>>> [4]
> >>>> https://bugs.launchpad.net/ironic-inspector/+milestone/2.0.0
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> __________________________________________________________________________
> >>>> OpenStack Development Mailing List (not for usage
> >>>> questions)
> >>>> Unsubscribe:
> >>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> >>>>
> >>>>
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> >>>>
> >>>>
> >>>>
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> >>>>
> >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> __________________________________________________________________________
> >>>> OpenStack Development Mailing List (not for usage
> questions)
> >>>> Unsubscribe:
> >>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> >>>>
> >>>>
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> >>>>
> >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> __________________________________________________________________________
> >>>> OpenStack Development Mailing List (not for usage questions)
> >>>> Unsubscribe:
> >>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> >>>>
> >>>>
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> __________________________________________________________________________
> >>>> OpenStack Development Mailing List (not for usage questions)
> >>>> Unsubscribe:
> >>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>>
> >>>
> >>>
> >>>
> >>>
> __________________________________________________________________________
> >>> OpenStack Development Mailing List (not for usage questions)
> >>> Unsubscribe:
> >>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >>
> >>
> __________________________________________________________________________
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >
> >
> >
> __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
More information about the OpenStack-dev
mailing list