<div dir="ltr">Dmitry, <div><div><br></div><div>If you chose to use Rally framework for testing there are 3 opportunities: </div><div><br></div><div> - Keep Rally plugins (tests) in separated tree </div><div> - Keep Rally plugins (tests) in your project tree </div><div> - Keep Rally plugins (tests) in Rally repo </div><div><br></div><div>Rally plugins can be used for all kinds of testing: (perf, scalability, load...) </div><div>so you are killing two birds with one stone. </div><div><br></div><div>P.S. I would imho prefer to keep all high quality plugins inside Rally repo to simplify operators life.. </div><div><br></div><div><br></div><div>Best regards,</div><div>Boris Pavlovic </div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jun 10, 2015 at 11:57 AM, Ken'ichi Ohmichi <span dir="ltr"><<a href="mailto:ken1ohmichi@gmail.com" target="_blank">ken1ohmichi@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">2015-06-10 16:48 GMT+09:00 Dmitry Tantsur <<a href="mailto:dtantsur@redhat.com">dtantsur@redhat.com</a>>:<br>
> On 06/10/2015 09:40 AM, Ken'ichi Ohmichi wrote:<br>
</span><span class="">>> To solve it, we have decided the scope of Tempest as the etherpad<br>
>> mentioned.<br>
>><br>
>>> Are there any hints now on where we can start with our integration tests?<br>
>><br>
>><br>
>> For the other projects, we are migrating the test framework of Tempest<br>
>> to tempest-lib which is a library.<br>
>> So each project can implement their own tests in each repository by<br>
>> using the test framework of tempest-lib.<br>
><br>
><br>
> So in my case we can start with putting test code to ironic-inspector tree<br>
> using tempest-lib, right?<br>
<br>
</span>Yeah, right.<br>
Neutron is already doing that.<br>
maybe neutron/tests/api/ of Neutron repository will be a hint for it.<br>
<span class=""><br>
> Will it be possible to run tests on Ironic as well using plugin from<br>
> ironic-inspector?<br>
<br>
</span>Yeah, it will be possible.<br>
but I'm guessing ironic-inspector is optional and Ironic should not<br>
depend on the gate test result of ironic-inspector.<br>
So maybe you just need to run Ironic tests on ironic-inspector gate<br>
tests, right?<br>
<span class=""><br>
>>> After a quick look at devstack-gate I got an impression that it's<br>
>>> expecting<br>
>>> tests as part of tempest:<br>
>>><br>
>>> <a href="https://github.com/openstack-infra/devstack-gate/blob/master/devstack-vm-gate.sh#L600" target="_blank">https://github.com/openstack-infra/devstack-gate/blob/master/devstack-vm-gate.sh#L600</a><br>
>>><br>
>>> Our final goal is to have devstack gate test for Ironic and Inspector<br>
>>> projects working together.<br>
>><br>
>><br>
>> We have discussed external interfaces of Tempest on the summit, so<br>
>> that Tempest gathers tests from each project repository and runs them<br>
>> at the same time.<br>
>> There is a qa-spec for <a href="https://review.openstack.org/#/c/184992/" target="_blank">https://review.openstack.org/#/c/184992/</a><br>
><br>
><br>
> Cool, thanks! Does it mean that devstack-gate will also be updated to allow<br>
> something like DEVSTACK_GATE_TEMPEST_PLUGINS="<a href="https://github.com/.." target="_blank">https://github.com/..</a>."?<br>
<br>
</span>Yeah, will be.<br>
The idea of this external interface is based on DevStack's one.<br>
I think we will be able to use it on the gate like that.<br>
<br>
Thanks<br>
<span class="HOEnZb"><font color="#888888">Ken'ichi Ohmichi<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
---<br>
<br>
>>> On 06/10/2015 08:07 AM, Yuiko Takada wrote:<br>
>>>><br>
>>>><br>
>>>> Hi, Dmitry,<br>
>>>><br>
>>>>      I guess the whole idea of new release models is NOT to tie projects<br>
>>>>      to each other any more except for The Big Release twice a year :)<br>
>>>> So<br>
>>>>      I think no, we don't need to. We still can do it, if we have<br>
>>>>      something to release by the time Ironic releases, but I suggest<br>
>>>>      deciding it on case-by-case basis.<br>
>>>><br>
>>>> OK, I see.<br>
>>>><br>
>>>> One more concern, about Tempest integration test which I will implement<br>
>>>> in V2.1.0,<br>
>>>> it seems like that we cannot add Ironic-inspector's tests into Tempest<br>
>>>> even if integration tests.<br>
>>>> Please see:<br>
>>>> <a href="https://etherpad.openstack.org/p/YVR-QA-in-the-big-tent" target="_blank">https://etherpad.openstack.org/p/YVR-QA-in-the-big-tent</a><br>
>>><br>
>>><br>
>>><br>
>>> Good catch. I guess the answer depends on where Ironic integration tests<br>
>>> are<br>
>>> going to live - we're going to live with them. Let me retarget this<br>
>>> thread<br>
>>> to a wider audience.<br>
>>><br>
>>>><br>
>>>> But I heard from you that Devananda thinks we need this in tempest<br>
>>>> itself. [3]<br>
>>>> Do you know something like current situation?<br>
>>>><br>
>>>><br>
>>>> Best Regards,<br>
>>>> Yuiko Takada<br>
>>>><br>
>>>> 2015-06-09 15:59 GMT+09:00 Dmitry Tantsur <<a href="mailto:dtantsur@redhat.com">dtantsur@redhat.com</a><br>
>>>> <mailto:<a href="mailto:dtantsur@redhat.com">dtantsur@redhat.com</a>>>:<br>
>>>><br>
>>>>      On 06/09/2015 03:49 AM, Yuiko Takada wrote:<br>
>>>><br>
>>>>          Hi, Dmitry,<br>
>>>><br>
>>>>          Thank you for notifying.<br>
>>>><br>
>>>>               I've updated our summit etherpad [3] with whatever<br>
>>>> priorities<br>
>>>> I<br>
>>>>               remembered, please have a look. I've also untargeted a few<br>
>>>>          things in<br>
>>>>               launchpad [4] (and will probably untarget more later on).<br>
>>>>          Please<br>
>>>>               assign yourself, if you want something done in this<br>
>>>> release<br>
>>>>          time frame.<br>
>>>><br>
>>>>          I've assigned one item to myself in [3], and also I added one<br>
>>>> BP<br>
>>>>          to [4],<br>
>>>>          so please take a look.<br>
>>>><br>
>>>> <a href="https://blueprints.launchpad.net/ironic-inspector/+spec/delete-db-api" target="_blank">https://blueprints.launchpad.net/ironic-inspector/+spec/delete-db-api</a><br>
>>>><br>
>>>><br>
>>>>      Looks good, though I don't think it's a big priority for 2.0.0.<br>
>>>>      Definitely worth doing for 2.1.0.<br>
>>>><br>
>>>>      Thanks for assigning for tempest work, that's definitely a priority<br>
>>>>      right now.<br>
>>>><br>
>>>><br>
>>>>          BTW, how do you think about Ironic-inspector's release model?<br>
>>>>          You wrote "Version released with Ironic Liberty" as<br>
>>>>          Ironic-inspector Version 2.1.0 in etherpad [3],<br>
>>>>          but as you know, Ironic's release model has changed to feature<br>
>>>>          releases.(right?)<br>
>>>>          Should we make our release model same as Ironic?<br>
>>>><br>
>>>><br>
>>>>      I guess the whole idea of new release models is NOT to tie projects<br>
>>>>      to each other any more except for The Big Release twice a year :)<br>
>>>> So<br>
>>>>      I think no, we don't need to. We still can do it, if we have<br>
>>>>      something to release by the time Ironic releases, but I suggest<br>
>>>>      deciding it on case-by-case basis.<br>
>>>><br>
>>>><br>
>>>><br>
>>>>          Best Regards,<br>
>>>>          Yuiko Takada(Inspector team member)<br>
>>>><br>
>>>>          2015-06-08 20:38 GMT+09:00 Dmitry Tantsur <<a href="mailto:dtantsur@redhat.com">dtantsur@redhat.com</a><br>
>>>>          <mailto:<a href="mailto:dtantsur@redhat.com">dtantsur@redhat.com</a>><br>
>>>>          <mailto:<a href="mailto:dtantsur@redhat.com">dtantsur@redhat.com</a> <mailto:<a href="mailto:dtantsur@redhat.com">dtantsur@redhat.com</a>>>>:<br>
>>>><br>
>>>><br>
>>>>               Hello, Inspector team!<br>
>>>><br>
>>>>               The renaming process is going pretty well, the last thing<br>
>>>>          we need to<br>
>>>>               do is to get Infra approval and actual rename [1][2].<br>
>>>><br>
>>>>               I'd like to allow people (e.g. myself) to start packaging<br>
>>>>          inspector<br>
>>>>               under it's new name, so I'd like to make 2.0.0 release as<br>
>>>>          soon as<br>
>>>>               possible (as opposed to scheduling it to particular date).<br>
>>>> All<br>
>>>>               breaking changes should land by this release - I don't<br>
>>>>          expect 3.0.0<br>
>>>>               soon :)<br>
>>>><br>
>>>>               I've updated our summit etherpad [3] with whatever<br>
>>>> priorities<br>
>>>> I<br>
>>>>               remembered, please have a look. I've also untargeted a few<br>
>>>>          things in<br>
>>>>               launchpad [4] (and will probably untarget more later on).<br>
>>>>          Please<br>
>>>>               assign yourself, if you want something done in this<br>
>>>> release<br>
>>>>          time frame.<br>
>>>><br>
>>>>               I would like 2.1.0 to be released with Ironic Liberty and<br>
>>>> be<br>
>>>>               properly supported.<br>
>>>><br>
>>>>               Let me know what you think.<br>
>>>><br>
>>>>               Cheers,<br>
>>>>               Dmitry<br>
>>>><br>
>>>>               [1] <a href="https://review.openstack.org/#/c/188030/" target="_blank">https://review.openstack.org/#/c/188030/</a><br>
>>>>               [2] <a href="https://review.openstack.org/#/c/188798/" target="_blank">https://review.openstack.org/#/c/188798/</a><br>
>>>>               [3]<br>
>>>> <a href="https://etherpad.openstack.org/p/liberty-ironic-discoverd" target="_blank">https://etherpad.openstack.org/p/liberty-ironic-discoverd</a><br>
>>>>               [4]<br>
>>>>          <a href="https://bugs.launchpad.net/ironic-inspector/+milestone/2.0.0" target="_blank">https://bugs.launchpad.net/ironic-inspector/+milestone/2.0.0</a><br>
>>>><br>
>>>><br>
>>>><br>
>>>><br>
>>>> __________________________________________________________________________<br>
>>>>               OpenStack Development Mailing List (not for usage<br>
>>>> questions)<br>
>>>>               Unsubscribe:<br>
>>>>          <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
>>>><br>
>>>> <<a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>><br>
>>>><br>
>>>><br>
>>>> <<a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>><br>
>>>><br>
>>>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
>>>><br>
>>>><br>
>>>><br>
>>>><br>
>>>><br>
>>>><br>
>>>> __________________________________________________________________________<br>
>>>>          OpenStack Development Mailing List (not for usage questions)<br>
>>>>          Unsubscribe:<br>
>>>>          <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
>>>><br>
>>>> <<a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>><br>
>>>><br>
>>>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
>>>><br>
>>>><br>
>>>><br>
>>>><br>
>>>><br>
>>>> __________________________________________________________________________<br>
>>>>      OpenStack Development Mailing List (not for usage questions)<br>
>>>>      Unsubscribe:<br>
>>>>      <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
>>>><br>
>>>> <<a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>><br>
>>>>      <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
>>>><br>
>>>><br>
>>>><br>
>>>><br>
>>>><br>
>>>> __________________________________________________________________________<br>
>>>> OpenStack Development Mailing List (not for usage questions)<br>
>>>> Unsubscribe:<br>
>>>> <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
>>>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
>>>><br>
>>><br>
>>><br>
>>><br>
>>> __________________________________________________________________________<br>
>>> OpenStack Development Mailing List (not for usage questions)<br>
>>> Unsubscribe:<br>
>>> <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
>>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
>><br>
>><br>
>> __________________________________________________________________________<br>
>> OpenStack Development Mailing List (not for usage questions)<br>
>> Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
>><br>
><br>
><br>
> __________________________________________________________________________<br>
> OpenStack Development Mailing List (not for usage questions)<br>
> Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</div></div></blockquote></div><br></div>