[openstack-dev] [QA] [Ironic] [Inspector] Where should integration tests for non-core projects live now? (Was: Toward 2.0.0 release)

Boris Pavlovic boris at pavlovic.me
Wed Jun 10 10:35:37 UTC 2015


Dmitry,

We introduced recently dsvm rally ironic job:
https://review.openstack.org/#/c/187997/

Now we are working on Rally tests for Ironic:
https://review.openstack.org/#/c/186064/

Don't hesitate to join us=)

Best regards,
Boris Pavlovic

On Wed, Jun 10, 2015 at 1:23 PM, Dmitry Tantsur <dtantsur at redhat.com> wrote:

> On 06/10/2015 11:57 AM, Boris Pavlovic wrote:
>
>> Dmitry,
>>
>> If you chose to use Rally framework for testing there are 3 opportunities:
>>
>>   - Keep Rally plugins (tests) in separated tree
>>   - Keep Rally plugins (tests) in your project tree
>>   - Keep Rally plugins (tests) in Rally repo
>>
>> Rally plugins can be used for all kinds of testing: (perf, scalability,
>> load...)
>> so you are killing two birds with one stone.
>>
>> P.S. I would imho prefer to keep all high quality plugins inside Rally
>> repo to simplify operators life..
>>
>
> Hi, that sounds interesting, I'll have a look.
>
> Note, however, that Inspector integration testing highly depends on Ironic
> one, so unless Ironic adapts/agrees to adapt Rally, it will be hard to
> Inspector to do it.
>
>
>>
>> Best regards,
>> Boris Pavlovic
>>
>> On Wed, Jun 10, 2015 at 11:57 AM, Ken'ichi Ohmichi
>> <ken1ohmichi at gmail.com <mailto:ken1ohmichi at gmail.com>> wrote:
>>
>>     2015-06-10 16:48 GMT+09:00 Dmitry Tantsur <dtantsur at redhat.com
>>     <mailto:dtantsur at redhat.com>>:
>>
>>     > On 06/10/2015 09:40 AM, Ken'ichi Ohmichi wrote:
>>     >> To solve it, we have decided the scope of Tempest as the etherpad
>>     >> mentioned.
>>     >>
>>     >>> Are there any hints now on where we can start with our
>> integration tests?
>>     >>
>>     >>
>>     >> For the other projects, we are migrating the test framework of
>> Tempest
>>     >> to tempest-lib which is a library.
>>     >> So each project can implement their own tests in each repository by
>>     >> using the test framework of tempest-lib.
>>     >
>>     >
>>     > So in my case we can start with putting test code to
>> ironic-inspector tree
>>     > using tempest-lib, right?
>>
>>     Yeah, right.
>>     Neutron is already doing that.
>>     maybe neutron/tests/api/ of Neutron repository will be a hint for it.
>>
>>     > Will it be possible to run tests on Ironic as well using plugin from
>>     > ironic-inspector?
>>
>>     Yeah, it will be possible.
>>     but I'm guessing ironic-inspector is optional and Ironic should not
>>     depend on the gate test result of ironic-inspector.
>>     So maybe you just need to run Ironic tests on ironic-inspector gate
>>     tests, right?
>>
>>     >>> After a quick look at devstack-gate I got an impression that it's
>>     >>> expecting
>>     >>> tests as part of tempest:
>>     >>>
>>     >>>
>> https://github.com/openstack-infra/devstack-gate/blob/master/devstack-vm-gate.sh#L600
>>     >>>
>>     >>> Our final goal is to have devstack gate test for Ironic and
>> Inspector
>>     >>> projects working together.
>>     >>
>>     >>
>>     >> We have discussed external interfaces of Tempest on the summit, so
>>     >> that Tempest gathers tests from each project repository and runs
>> them
>>     >> at the same time.
>>     >> There is a qa-spec forhttps://review.openstack.org/#/c/184992/
>>
>>     >
>>     >
>>     > Cool, thanks! Does it mean that devstack-gate will also be updated
>> to allow
>>     > something like DEVSTACK_GATE_TEMPEST_PLUGINS="https://github.com/..
>> ."?
>>
>>     Yeah, will be.
>>     The idea of this external interface is based on DevStack's one.
>>     I think we will be able to use it on the gate like that.
>>
>>     Thanks
>>     Ken'ichi Ohmichi
>>
>>     ---
>>
>>      >>> On 06/10/2015 08:07 AM, Yuiko Takada wrote:
>>      >>>>
>>      >>>>
>>      >>>> Hi, Dmitry,
>>      >>>>
>>      >>>>      I guess the whole idea of new release models is NOT to
>>     tie projects
>>      >>>>      to each other any more except for The Big Release twice a
>>     year :)
>>      >>>> So
>>      >>>>      I think no, we don't need to. We still can do it, if we
>> have
>>      >>>>      something to release by the time Ironic releases, but I
>>     suggest
>>      >>>>      deciding it on case-by-case basis.
>>      >>>>
>>      >>>> OK, I see.
>>      >>>>
>>      >>>> One more concern, about Tempest integration test which I will
>>     implement
>>      >>>> in V2.1.0,
>>      >>>> it seems like that we cannot add Ironic-inspector's tests into
>>     Tempest
>>      >>>> even if integration tests.
>>      >>>> Please see:
>>      >>>> https://etherpad.openstack.org/p/YVR-QA-in-the-big-tent
>>      >>>
>>      >>>
>>      >>>
>>      >>> Good catch. I guess the answer depends on where Ironic
>>     integration tests
>>      >>> are
>>      >>> going to live - we're going to live with them. Let me retarget
>> this
>>      >>> thread
>>      >>> to a wider audience.
>>      >>>
>>      >>>>
>>      >>>> But I heard from you that Devananda thinks we need this in
>> tempest
>>      >>>> itself. [3]
>>      >>>> Do you know something like current situation?
>>      >>>>
>>      >>>>
>>      >>>> Best Regards,
>>      >>>> Yuiko Takada
>>      >>>>
>>      >>>> 2015-06-09 15:59 GMT+09:00 Dmitry Tantsur <dtantsur at redhat.com
>>     <mailto:dtantsur at redhat.com>
>>      >>>> <mailto:dtantsur at redhat.com <mailto:dtantsur at redhat.com>>>:
>>
>>      >>>>
>>      >>>>      On 06/09/2015 03:49 AM, Yuiko Takada wrote:
>>      >>>>
>>      >>>>          Hi, Dmitry,
>>      >>>>
>>      >>>>          Thank you for notifying.
>>      >>>>
>>      >>>>               I've updated our summit etherpad [3] with whatever
>>      >>>> priorities
>>      >>>> I
>>      >>>>               remembered, please have a look. I've also
>>     untargeted a few
>>      >>>>          things in
>>      >>>>               launchpad [4] (and will probably untarget more
>>     later on).
>>      >>>>          Please
>>      >>>>               assign yourself, if you want something done in
>> this
>>      >>>> release
>>      >>>>          time frame.
>>      >>>>
>>      >>>>          I've assigned one item to myself in [3], and also I
>>     added one
>>      >>>> BP
>>      >>>>          to [4],
>>      >>>>          so please take a look.
>>      >>>>
>>      >>>>
>>     https://blueprints.launchpad.net/ironic-inspector/+spec/delete-db-api
>>      >>>>
>>      >>>>
>>      >>>>      Looks good, though I don't think it's a big priority for
>>     2.0.0.
>>      >>>>      Definitely worth doing for 2.1.0.
>>      >>>>
>>      >>>>      Thanks for assigning for tempest work, that's definitely
>>     a priority
>>      >>>>      right now.
>>      >>>>
>>      >>>>
>>      >>>>          BTW, how do you think about Ironic-inspector's
>>     release model?
>>      >>>>          You wrote "Version released with Ironic Liberty" as
>>      >>>>          Ironic-inspector Version 2.1.0 in etherpad [3],
>>      >>>>          but as you know, Ironic's release model has changed
>>     to feature
>>      >>>>          releases.(right?)
>>      >>>>          Should we make our release model same as Ironic?
>>      >>>>
>>      >>>>
>>      >>>>      I guess the whole idea of new release models is NOT to
>>     tie projects
>>      >>>>      to each other any more except for The Big Release twice a
>>     year :)
>>      >>>> So
>>      >>>>      I think no, we don't need to. We still can do it, if we
>> have
>>      >>>>      something to release by the time Ironic releases, but I
>>     suggest
>>      >>>>      deciding it on case-by-case basis.
>>      >>>>
>>      >>>>
>>      >>>>
>>      >>>>          Best Regards,
>>      >>>>          Yuiko Takada(Inspector team member)
>>      >>>>
>>      >>>>          2015-06-08 20:38 GMT+09:00 Dmitry Tantsur
>>     <dtantsur at redhat.com <mailto:dtantsur at redhat.com>
>>      >>>>          <mailto:dtantsur at redhat.com <mailto:
>> dtantsur at redhat.com>>
>>      >>>>          <mailto:dtantsur at redhat.com
>>     <mailto:dtantsur at redhat.com> <mailto:dtantsur at redhat.com
>>     <mailto:dtantsur at redhat.com>>>>:
>>      >>>>
>>      >>>>
>>      >>>>               Hello, Inspector team!
>>      >>>>
>>      >>>>               The renaming process is going pretty well, the
>>     last thing
>>      >>>>          we need to
>>      >>>>               do is to get Infra approval and actual rename
>>     [1][2].
>>      >>>>
>>      >>>>               I'd like to allow people (e.g. myself) to start
>>     packaging
>>      >>>>          inspector
>>      >>>>               under it's new name, so I'd like to make 2.0.0
>>     release as
>>      >>>>          soon as
>>      >>>>               possible (as opposed to scheduling it to
>>     particular date).
>>      >>>> All
>>      >>>>               breaking changes should land by this release - I
>>     don't
>>      >>>>          expect 3.0.0
>>      >>>>               soon :)
>>      >>>>
>>      >>>>               I've updated our summit etherpad [3] with whatever
>>      >>>> priorities
>>      >>>> I
>>      >>>>               remembered, please have a look. I've also
>>     untargeted a few
>>      >>>>          things in
>>      >>>>               launchpad [4] (and will probably untarget more
>>     later on).
>>      >>>>          Please
>>      >>>>               assign yourself, if you want something done in
>> this
>>      >>>> release
>>      >>>>          time frame.
>>      >>>>
>>      >>>>               I would like 2.1.0 to be released with Ironic
>>     Liberty and
>>      >>>> be
>>      >>>>               properly supported.
>>      >>>>
>>      >>>>               Let me know what you think.
>>      >>>>
>>      >>>>               Cheers,
>>      >>>>               Dmitry
>>      >>>>
>>      >>>>               [1] https://review.openstack.org/#/c/188030/
>>      >>>>               [2] https://review.openstack.org/#/c/188798/
>>      >>>>               [3]
>>      >>>> https://etherpad.openstack.org/p/liberty-ironic-discoverd
>>      >>>>               [4]
>>      >>>> https://bugs.launchpad.net/ironic-inspector/+milestone/2.0.0
>>      >>>>
>>      >>>>
>>      >>>>
>>      >>>>
>>      >>>>
>>
>> __________________________________________________________________________
>>      >>>>               OpenStack Development Mailing List (not for usage
>>      >>>> questions)
>>      >>>>               Unsubscribe:
>>      >>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>     <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>> >
>>      >>>>
>>      >>>>
>>     <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>> >
>>      >>>>
>>      >>>>
>>      >>>>
>>     <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>> >
>>      >>>>
>>      >>>>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>      >>>>
>>      >>>>
>>      >>>>
>>      >>>>
>>      >>>>
>>      >>>>
>>      >>>>
>>
>> __________________________________________________________________________
>>      >>>>          OpenStack Development Mailing List (not for usage
>>     questions)
>>      >>>>          Unsubscribe:
>>      >>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>     <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>> >
>>      >>>>
>>      >>>>
>>     <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>> >
>>      >>>>
>>      >>>>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>      >>>>
>>      >>>>
>>      >>>>
>>      >>>>
>>      >>>>
>>      >>>>
>>
>> __________________________________________________________________________
>>      >>>>      OpenStack Development Mailing List (not for usage
>> questions)
>>      >>>>      Unsubscribe:
>>      >>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>     <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>> >
>>      >>>>
>>      >>>>
>>     <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>> >
>>      >>>>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>      >>>>
>>      >>>>
>>      >>>>
>>      >>>>
>>      >>>>
>>      >>>>
>>
>> __________________________________________________________________________
>>      >>>> OpenStack Development Mailing List (not for usage questions)
>>      >>>> Unsubscribe:
>>      >>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>     <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>> >
>>      >>>>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>      >>>>
>>      >>>
>>      >>>
>>      >>>
>>      >>>
>>
>> __________________________________________________________________________
>>      >>> OpenStack Development Mailing List (not for usage questions)
>>      >>> Unsubscribe:
>>      >>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>     <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>> >
>>      >>>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>      >>
>>      >>
>>      >>
>>
>> __________________________________________________________________________
>>      >> OpenStack Development Mailing List (not for usage questions)
>>      >> Unsubscribe:
>>     OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>     <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>> >
>>      >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>      >>
>>      >
>>      >
>>      >
>>
>> __________________________________________________________________________
>>      > OpenStack Development Mailing List (not for usage questions)
>>      > Unsubscribe:
>>     OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>     <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>> >
>>      > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>> __________________________________________________________________________
>>     OpenStack Development Mailing List (not for usage questions)
>>     Unsubscribe:
>>     OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>     <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>> >
>>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150610/20a6373a/attachment.html>


More information about the OpenStack-dev mailing list