[openstack-dev] [QA] [Ironic] [Inspector] Where should integration tests for non-core projects live now? (Was: Toward 2.0.0 release)
Boris Pavlovic
boris at pavlovic.me
Wed Jun 10 10:35:37 UTC 2015
Dmitry,
We introduced recently dsvm rally ironic job:
https://review.openstack.org/#/c/187997/
Now we are working on Rally tests for Ironic:
https://review.openstack.org/#/c/186064/
Don't hesitate to join us=)
Best regards,
Boris Pavlovic
On Wed, Jun 10, 2015 at 1:23 PM, Dmitry Tantsur <dtantsur at redhat.com> wrote:
> On 06/10/2015 11:57 AM, Boris Pavlovic wrote:
>
>> Dmitry,
>>
>> If you chose to use Rally framework for testing there are 3 opportunities:
>>
>> - Keep Rally plugins (tests) in separated tree
>> - Keep Rally plugins (tests) in your project tree
>> - Keep Rally plugins (tests) in Rally repo
>>
>> Rally plugins can be used for all kinds of testing: (perf, scalability,
>> load...)
>> so you are killing two birds with one stone.
>>
>> P.S. I would imho prefer to keep all high quality plugins inside Rally
>> repo to simplify operators life..
>>
>
> Hi, that sounds interesting, I'll have a look.
>
> Note, however, that Inspector integration testing highly depends on Ironic
> one, so unless Ironic adapts/agrees to adapt Rally, it will be hard to
> Inspector to do it.
>
>
>>
>> Best regards,
>> Boris Pavlovic
>>
>> On Wed, Jun 10, 2015 at 11:57 AM, Ken'ichi Ohmichi
>> <ken1ohmichi at gmail.com <mailto:ken1ohmichi at gmail.com>> wrote:
>>
>> 2015-06-10 16:48 GMT+09:00 Dmitry Tantsur <dtantsur at redhat.com
>> <mailto:dtantsur at redhat.com>>:
>>
>> > On 06/10/2015 09:40 AM, Ken'ichi Ohmichi wrote:
>> >> To solve it, we have decided the scope of Tempest as the etherpad
>> >> mentioned.
>> >>
>> >>> Are there any hints now on where we can start with our
>> integration tests?
>> >>
>> >>
>> >> For the other projects, we are migrating the test framework of
>> Tempest
>> >> to tempest-lib which is a library.
>> >> So each project can implement their own tests in each repository by
>> >> using the test framework of tempest-lib.
>> >
>> >
>> > So in my case we can start with putting test code to
>> ironic-inspector tree
>> > using tempest-lib, right?
>>
>> Yeah, right.
>> Neutron is already doing that.
>> maybe neutron/tests/api/ of Neutron repository will be a hint for it.
>>
>> > Will it be possible to run tests on Ironic as well using plugin from
>> > ironic-inspector?
>>
>> Yeah, it will be possible.
>> but I'm guessing ironic-inspector is optional and Ironic should not
>> depend on the gate test result of ironic-inspector.
>> So maybe you just need to run Ironic tests on ironic-inspector gate
>> tests, right?
>>
>> >>> After a quick look at devstack-gate I got an impression that it's
>> >>> expecting
>> >>> tests as part of tempest:
>> >>>
>> >>>
>> https://github.com/openstack-infra/devstack-gate/blob/master/devstack-vm-gate.sh#L600
>> >>>
>> >>> Our final goal is to have devstack gate test for Ironic and
>> Inspector
>> >>> projects working together.
>> >>
>> >>
>> >> We have discussed external interfaces of Tempest on the summit, so
>> >> that Tempest gathers tests from each project repository and runs
>> them
>> >> at the same time.
>> >> There is a qa-spec forhttps://review.openstack.org/#/c/184992/
>>
>> >
>> >
>> > Cool, thanks! Does it mean that devstack-gate will also be updated
>> to allow
>> > something like DEVSTACK_GATE_TEMPEST_PLUGINS="https://github.com/..
>> ."?
>>
>> Yeah, will be.
>> The idea of this external interface is based on DevStack's one.
>> I think we will be able to use it on the gate like that.
>>
>> Thanks
>> Ken'ichi Ohmichi
>>
>> ---
>>
>> >>> On 06/10/2015 08:07 AM, Yuiko Takada wrote:
>> >>>>
>> >>>>
>> >>>> Hi, Dmitry,
>> >>>>
>> >>>> I guess the whole idea of new release models is NOT to
>> tie projects
>> >>>> to each other any more except for The Big Release twice a
>> year :)
>> >>>> So
>> >>>> I think no, we don't need to. We still can do it, if we
>> have
>> >>>> something to release by the time Ironic releases, but I
>> suggest
>> >>>> deciding it on case-by-case basis.
>> >>>>
>> >>>> OK, I see.
>> >>>>
>> >>>> One more concern, about Tempest integration test which I will
>> implement
>> >>>> in V2.1.0,
>> >>>> it seems like that we cannot add Ironic-inspector's tests into
>> Tempest
>> >>>> even if integration tests.
>> >>>> Please see:
>> >>>> https://etherpad.openstack.org/p/YVR-QA-in-the-big-tent
>> >>>
>> >>>
>> >>>
>> >>> Good catch. I guess the answer depends on where Ironic
>> integration tests
>> >>> are
>> >>> going to live - we're going to live with them. Let me retarget
>> this
>> >>> thread
>> >>> to a wider audience.
>> >>>
>> >>>>
>> >>>> But I heard from you that Devananda thinks we need this in
>> tempest
>> >>>> itself. [3]
>> >>>> Do you know something like current situation?
>> >>>>
>> >>>>
>> >>>> Best Regards,
>> >>>> Yuiko Takada
>> >>>>
>> >>>> 2015-06-09 15:59 GMT+09:00 Dmitry Tantsur <dtantsur at redhat.com
>> <mailto:dtantsur at redhat.com>
>> >>>> <mailto:dtantsur at redhat.com <mailto:dtantsur at redhat.com>>>:
>>
>> >>>>
>> >>>> On 06/09/2015 03:49 AM, Yuiko Takada wrote:
>> >>>>
>> >>>> Hi, Dmitry,
>> >>>>
>> >>>> Thank you for notifying.
>> >>>>
>> >>>> I've updated our summit etherpad [3] with whatever
>> >>>> priorities
>> >>>> I
>> >>>> remembered, please have a look. I've also
>> untargeted a few
>> >>>> things in
>> >>>> launchpad [4] (and will probably untarget more
>> later on).
>> >>>> Please
>> >>>> assign yourself, if you want something done in
>> this
>> >>>> release
>> >>>> time frame.
>> >>>>
>> >>>> I've assigned one item to myself in [3], and also I
>> added one
>> >>>> BP
>> >>>> to [4],
>> >>>> so please take a look.
>> >>>>
>> >>>>
>> https://blueprints.launchpad.net/ironic-inspector/+spec/delete-db-api
>> >>>>
>> >>>>
>> >>>> Looks good, though I don't think it's a big priority for
>> 2.0.0.
>> >>>> Definitely worth doing for 2.1.0.
>> >>>>
>> >>>> Thanks for assigning for tempest work, that's definitely
>> a priority
>> >>>> right now.
>> >>>>
>> >>>>
>> >>>> BTW, how do you think about Ironic-inspector's
>> release model?
>> >>>> You wrote "Version released with Ironic Liberty" as
>> >>>> Ironic-inspector Version 2.1.0 in etherpad [3],
>> >>>> but as you know, Ironic's release model has changed
>> to feature
>> >>>> releases.(right?)
>> >>>> Should we make our release model same as Ironic?
>> >>>>
>> >>>>
>> >>>> I guess the whole idea of new release models is NOT to
>> tie projects
>> >>>> to each other any more except for The Big Release twice a
>> year :)
>> >>>> So
>> >>>> I think no, we don't need to. We still can do it, if we
>> have
>> >>>> something to release by the time Ironic releases, but I
>> suggest
>> >>>> deciding it on case-by-case basis.
>> >>>>
>> >>>>
>> >>>>
>> >>>> Best Regards,
>> >>>> Yuiko Takada(Inspector team member)
>> >>>>
>> >>>> 2015-06-08 20:38 GMT+09:00 Dmitry Tantsur
>> <dtantsur at redhat.com <mailto:dtantsur at redhat.com>
>> >>>> <mailto:dtantsur at redhat.com <mailto:
>> dtantsur at redhat.com>>
>> >>>> <mailto:dtantsur at redhat.com
>> <mailto:dtantsur at redhat.com> <mailto:dtantsur at redhat.com
>> <mailto:dtantsur at redhat.com>>>>:
>> >>>>
>> >>>>
>> >>>> Hello, Inspector team!
>> >>>>
>> >>>> The renaming process is going pretty well, the
>> last thing
>> >>>> we need to
>> >>>> do is to get Infra approval and actual rename
>> [1][2].
>> >>>>
>> >>>> I'd like to allow people (e.g. myself) to start
>> packaging
>> >>>> inspector
>> >>>> under it's new name, so I'd like to make 2.0.0
>> release as
>> >>>> soon as
>> >>>> possible (as opposed to scheduling it to
>> particular date).
>> >>>> All
>> >>>> breaking changes should land by this release - I
>> don't
>> >>>> expect 3.0.0
>> >>>> soon :)
>> >>>>
>> >>>> I've updated our summit etherpad [3] with whatever
>> >>>> priorities
>> >>>> I
>> >>>> remembered, please have a look. I've also
>> untargeted a few
>> >>>> things in
>> >>>> launchpad [4] (and will probably untarget more
>> later on).
>> >>>> Please
>> >>>> assign yourself, if you want something done in
>> this
>> >>>> release
>> >>>> time frame.
>> >>>>
>> >>>> I would like 2.1.0 to be released with Ironic
>> Liberty and
>> >>>> be
>> >>>> properly supported.
>> >>>>
>> >>>> Let me know what you think.
>> >>>>
>> >>>> Cheers,
>> >>>> Dmitry
>> >>>>
>> >>>> [1] https://review.openstack.org/#/c/188030/
>> >>>> [2] https://review.openstack.org/#/c/188798/
>> >>>> [3]
>> >>>> https://etherpad.openstack.org/p/liberty-ironic-discoverd
>> >>>> [4]
>> >>>> https://bugs.launchpad.net/ironic-inspector/+milestone/2.0.0
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>>
>> __________________________________________________________________________
>> >>>> OpenStack Development Mailing List (not for usage
>> >>>> questions)
>> >>>> Unsubscribe:
>> >>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>> >
>> >>>>
>> >>>>
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>> >
>> >>>>
>> >>>>
>> >>>>
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>> >
>> >>>>
>> >>>>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>>
>> __________________________________________________________________________
>> >>>> OpenStack Development Mailing List (not for usage
>> questions)
>> >>>> Unsubscribe:
>> >>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>> >
>> >>>>
>> >>>>
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>> >
>> >>>>
>> >>>>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>>
>> __________________________________________________________________________
>> >>>> OpenStack Development Mailing List (not for usage
>> questions)
>> >>>> Unsubscribe:
>> >>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>> >
>> >>>>
>> >>>>
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>> >
>> >>>>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>>
>> __________________________________________________________________________
>> >>>> OpenStack Development Mailing List (not for usage questions)
>> >>>> Unsubscribe:
>> >>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>> >
>> >>>>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>>
>>
>> __________________________________________________________________________
>> >>> OpenStack Development Mailing List (not for usage questions)
>> >>> Unsubscribe:
>> >>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>> >
>> >>>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >>
>> >>
>> >>
>>
>> __________________________________________________________________________
>> >> OpenStack Development Mailing List (not for usage questions)
>> >> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>> >
>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >>
>> >
>> >
>> >
>>
>> __________________________________________________________________________
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>> >
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>> >
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150610/20a6373a/attachment.html>
More information about the OpenStack-dev
mailing list