<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Tue, May 2, 2017 at 5:27 AM, MONTEIRO, FELIPE C <span dir="ltr"><<a href="mailto:fm577c@att.com" target="_blank">fm577c@att.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Murano currently uses the Tempest orchestration client for its scenario Tempest tests [0], which are not turned on by default in the Murano Tempest gate due to resource constraints.<br>
<br>
However, I'm hesitant to switch to Heat's testing client, because it is not a Tempest client, but rather the python-heatclient. I would like to know whether there are plans to change this to a Tempest-based client? <br></blockquote><div><br></div><div>There are no plans to switch the heat integration/functional tests to using the tempest based client. The heat tests will use heatclient for most tests, and gabbi for testing the REST API.</div><div><br></div><div>Since you're testing Murano rather than the Heat API, I think converting your tests to heatclient would be reasonable.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
[0] <a href="https://github.com/openstack/murano/blob/master/murano_tempest_tests/tests/scenario/application_catalog/base.py#L100" rel="noreferrer" target="_blank">https://github.com/openstack/<wbr>murano/blob/master/murano_<wbr>tempest_tests/tests/scenario/<wbr>application_catalog/base.py#<wbr>L100</a><br>
[1] <a href="https://github.com/openstack/heat/blob/master/heat_integrationtests/common/clients.py#L120" rel="noreferrer" target="_blank">https://github.com/openstack/<wbr>heat/blob/master/heat_<wbr>integrationtests/common/<wbr>clients.py#L120</a><br>
<br>
Felipe<br>
<div><div class="h5"><br>
-----Original Message-----<br>
From: Ghanshyam Mann [mailto:<a href="mailto:ghanshyammann@gmail.com">ghanshyammann@gmail.<wbr>com</a>]<br>
Sent: Sunday, April 30, 2017 1:53 AM<br>
To: OpenStack Development Mailing List (not for usage questions) <<a href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.<wbr>openstack.org</a>><br>
Subject: Re: [openstack-dev] [qa][heat][murano][daisycloud] Removing Heat support from Tempest<br>
<br>
On Fri, Apr 28, 2017 at 5:47 PM, Andrea Frittoli<br>
<<a href="mailto:andrea.frittoli@gmail.com">andrea.frittoli@gmail.com</a>> wrote:<br>
><br>
><br>
> On Fri, Apr 28, 2017 at 10:29 AM Rabi Mishra <<a href="mailto:ramishra@redhat.com">ramishra@redhat.com</a>> wrote:<br>
>><br>
>> On Thu, Apr 27, 2017 at 3:55 PM, Andrea Frittoli<br>
>> <<a href="mailto:andrea.frittoli@gmail.com">andrea.frittoli@gmail.com</a>> wrote:<br>
>>><br>
>>> Dear stackers,<br>
>>><br>
>>> starting in the Liberty cycle Tempest has defined a set of projects which<br>
>>> are in scope for direct<br>
>>> testing in Tempest [0]. The current list includes keystone, nova, glance,<br>
>>> swift, cinder and neutron.<br>
>>> All other projects can use the same Tempest testing infrastructure (or<br>
>>> parts of it) by taking advantage<br>
>>> the Tempest plugin and stable interfaces.<br>
>>><br>
>>> Tempest currently hosts a set of API tests as well as a service client<br>
>>> for the Heat project.<br>
>>> The Heat service client is used by the tests in Tempest, which run in<br>
>>> Heat gate as part of the grenade<br>
>>> job, as well as in the Tempest gate (check pipeline) as part of the<br>
>>> layer4 job.<br>
>>> According to code search [3] the Heat service client is also used by<br>
>>> Murano and Daisycore.<br>
>><br>
>><br>
>> For the heat grenade job, I've proposed two patches.<br>
>><br>
>> 1. To run heat tree gabbi api tests as part of grenade 'post-upgrade'<br>
>> phase<br>
>><br>
</div></div>>> <a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__review.openstack.org_-23_c_460542_&d=DwIGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=X4GwEru-SJ9DRnCxhze-aw&m=aN-OTm6qpDxNIXC86mUeowDuZe9O-NeCWHJdSvrVsYA&s=d2pZwZ8xKsFLHxQ0YNiM4itJjUHzgE0ibHNu7v28mXM&e=" rel="noreferrer" target="_blank">https://urldefense.proofpoint.<wbr>com/v2/url?u=https-3A__review.<wbr>openstack.org_-23_c_460542_&d=<wbr>DwIGaQ&c=LFYZ-o9_<wbr>HUMeMTSQicvjIg&r=X4GwEru-<wbr>SJ9DRnCxhze-aw&m=aN-<wbr>OTm6qpDxNIXC86mUeowDuZe9O-<wbr>NeCWHJdSvrVsYA&s=<wbr>d2pZwZ8xKsFLHxQ0YNiM4itJjUHzgE<wbr>0ibHNu7v28mXM&e=</a><br>
<span class="">>><br>
>> 2. To remove tempest tests from the grenade job<br>
>><br>
</span>>> <a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__review.openstack.org_-23_c_460810_&d=DwIGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=X4GwEru-SJ9DRnCxhze-aw&m=aN-OTm6qpDxNIXC86mUeowDuZe9O-NeCWHJdSvrVsYA&s=07__zljUdvdtD_K5ltoKwdjaBwrs0fYJKaXSr93AAiU&e=" rel="noreferrer" target="_blank">https://urldefense.proofpoint.<wbr>com/v2/url?u=https-3A__review.<wbr>openstack.org_-23_c_460810_&d=<wbr>DwIGaQ&c=LFYZ-o9_<wbr>HUMeMTSQicvjIg&r=X4GwEru-<wbr>SJ9DRnCxhze-aw&m=aN-<wbr>OTm6qpDxNIXC86mUeowDuZe9O-<wbr>NeCWHJdSvrVsYA&s=07__<wbr>zljUdvdtD_<wbr>K5ltoKwdjaBwrs0fYJKaXSr93AAiU&<wbr>e=</a><br>
<div><div class="h5">>><br>
>><br>
>>><br>
>>> I proposed a patch to Tempest to start the deprecation counter for Heat /<br>
>>> orchestration related<br>
>>> configuration items in Tempest [4], and I would like to make sure that<br>
>>> all tests and the service client<br>
>>> either find a new home outside of Tempest, or are removed, by the end the<br>
>>> Pike cycle at the latest.<br>
>>><br>
>>> Heat has in-tree integration tests and Gabbi based API tests, but I don't<br>
>>> know if those provide<br>
>>> enough coverage to replace the tests on Tempest side.<br>
>>><br>
>><br>
>> Yes, the heat gabbi api tests do not yet have the same coverage as the<br>
>> tempest tree api tests (lacks tests using nova, neutron and swift<br>
>> resources),  but I think that should not stop us from *not* running the<br>
>> tempest tests in the grenade job.<br>
>><br>
>> I also don't know if the tempest tree heat tests are used by any other<br>
>> upstream/downstream jobs. We could surely add more tests to bridge the gap.<br>
>><br>
>> Also, It's possible to run the heat integration tests (we've enough<br>
>> coverage there) with tempest plugin after doing some initial setup, as we do<br>
>> in all our dsvm gate jobs.<br>
>><br>
>>> It would propose to move tests and client to a Tempest plugin owned /<br>
>>> maintained by<br>
>>> the Heat team, so that the Heat team can have full flexibility in<br>
>>> consolidating their integration<br>
>>> tests. For Murano and Daisycloud - and any other team that may want to<br>
>>> use the Heat service<br>
>>> client in their tests, even if the client is removed from Tempest, it<br>
>>> would still be available via<br>
>>> the Heat Tempest plugin. As long as the plugin implements the service<br>
>>> client interface,<br>
>>> the Heat service client will register automatically in the service client<br>
>>> manager and be available<br>
>>> for use as today.<br>
>>><br>
>><br>
>> if I understand correctly, you're proposing moving the existing tempest<br>
>> tests and service clients to a separate repo managed by heat team. Though<br>
>> that would be collective decision, I'm not sure that's something I would<br>
>> like to do. To start with we may look at adding some of the missing pieces<br>
>> in heat tree itself.<br>
><br>
><br>
> I'm proposing to move tests and the service client outside of tempest to a<br>
> new home.<br>
><br>
> I also suggested that the new home could be a dedicate repo, since that<br>
> would allow you to maintain the<br>
> current branchless nature of those tests. A more detailed discussion about<br>
> the topic can be found<br>
> in the corresponding proposed queens goal [5],<br>
><br>
> Using a dedicated repo *is not* a precondition for moving tests and service<br>
> client out of Tempest.<br>
><br>
> andrea<br>
><br>
<br>
Other than grenade and layer4 job on Tempest, do we run heat tests<br>
anywhere else? With Rabi's proposed changes on grenade, If tempest<br>
tests are not going to run on gate any more and heat team is going to<br>
improve the heat in-tree integration tests coverage, then we can just<br>
remove tests and corresponding config options as Andrea proposed.<br>
<br>
Otherwise, IMO it should be done with separate repo to avoid more work<br>
once proposed TC goal is approved. But we should make decision on<br>
tempest heat tests now as its been long time those are maintained in<br>
Tempest.<br>
<br>
</div></div>> [5] <a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__review.openstack.org_-23_c_369749_&d=DwIGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=X4GwEru-SJ9DRnCxhze-aw&m=aN-OTm6qpDxNIXC86mUeowDuZe9O-NeCWHJdSvrVsYA&s=8C3kPSTqpT4tFcIUEC1bYVUgAPC-W08UECtKEAck6mc&e=" rel="noreferrer" target="_blank">https://urldefense.proofpoint.<wbr>com/v2/url?u=https-3A__review.<wbr>openstack.org_-23_c_369749_&d=<wbr>DwIGaQ&c=LFYZ-o9_<wbr>HUMeMTSQicvjIg&r=X4GwEru-<wbr>SJ9DRnCxhze-aw&m=aN-<wbr>OTm6qpDxNIXC86mUeowDuZe9O-<wbr>NeCWHJdSvrVsYA&s=<wbr>8C3kPSTqpT4tFcIUEC1bYVUgAPC-<wbr>W08UECtKEAck6mc&e=</a><br>
><br>
>><br>
>><br>
>>> Andrea Frittoli (andreaf)<br>
>>><br>
>>> [0]<br>
>>> <a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.openstack.org_developer_tempest_test-5Fremoval.html-23tempest-2Dscope&d=DwIGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=X4GwEru-SJ9DRnCxhze-aw&m=aN-OTm6qpDxNIXC86mUeowDuZe9O-NeCWHJdSvrVsYA&s=zR8OxN2YWCkTSqwOdXO86ijrTvhfcrz-jTcYDWcerSI&e=" rel="noreferrer" target="_blank">https://urldefense.proofpoint.<wbr>com/v2/url?u=https-3A__docs.<wbr>openstack.org_developer_<wbr>tempest_test-5Fremoval.html-<wbr>23tempest-2Dscope&d=DwIGaQ&c=<wbr>LFYZ-o9_HUMeMTSQicvjIg&r=<wbr>X4GwEru-SJ9DRnCxhze-aw&m=aN-<wbr>OTm6qpDxNIXC86mUeowDuZe9O-<wbr>NeCWHJdSvrVsYA&s=<wbr>zR8OxN2YWCkTSqwOdXO86ijrTvhfcr<wbr>z-jTcYDWcerSI&e=</a><br>
>>> [1] <a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.openstack.org_developer_tempest_plugin.html&d=DwIGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=X4GwEru-SJ9DRnCxhze-aw&m=aN-OTm6qpDxNIXC86mUeowDuZe9O-NeCWHJdSvrVsYA&s=JTzZ4W8taIX9LY35e5yEZLFpBJT0NfHlkQ9Fdm875lE&e=" rel="noreferrer" target="_blank">https://urldefense.proofpoint.<wbr>com/v2/url?u=https-3A__docs.<wbr>openstack.org_developer_<wbr>tempest_plugin.html&d=DwIGaQ&<wbr>c=LFYZ-o9_HUMeMTSQicvjIg&r=<wbr>X4GwEru-SJ9DRnCxhze-aw&m=aN-<wbr>OTm6qpDxNIXC86mUeowDuZe9O-<wbr>NeCWHJdSvrVsYA&s=<wbr>JTzZ4W8taIX9LY35e5yEZLFpBJT0Nf<wbr>HlkQ9Fdm875lE&e=</a><br>
>>> [2] <a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.openstack.org_developer_tempest_library.html&d=DwIGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=X4GwEru-SJ9DRnCxhze-aw&m=aN-OTm6qpDxNIXC86mUeowDuZe9O-NeCWHJdSvrVsYA&s=b4aI4XWUkF8-yJy_EGtTpAJ9O47LQSFzz6oplawJuE4&e=" rel="noreferrer" target="_blank">https://urldefense.proofpoint.<wbr>com/v2/url?u=https-3A__docs.<wbr>openstack.org_developer_<wbr>tempest_library.html&d=DwIGaQ&<wbr>c=LFYZ-o9_HUMeMTSQicvjIg&r=<wbr>X4GwEru-SJ9DRnCxhze-aw&m=aN-<wbr>OTm6qpDxNIXC86mUeowDuZe9O-<wbr>NeCWHJdSvrVsYA&s=b4aI4XWUkF8-<wbr>yJy_<wbr>EGtTpAJ9O47LQSFzz6oplawJuE4&e=</a><br>
>>> [3]<br>
>>> <a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__codesearch.openstack.org_-3Fq-3Dself.orchestration-5Fclient-26i-3Dnope-26files-3D-26repos-3D&d=DwIGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=X4GwEru-SJ9DRnCxhze-aw&m=aN-OTm6qpDxNIXC86mUeowDuZe9O-NeCWHJdSvrVsYA&s=Lw4Nep7Zyf-sUfbKo8DeOjVzVMWKxOwfD3G2MzZXjAQ&e=" rel="noreferrer" target="_blank">https://urldefense.proofpoint.<wbr>com/v2/url?u=http-3A__<wbr>codesearch.openstack.org_-3Fq-<wbr>3Dself.orchestration-5Fclient-<wbr>26i-3Dnope-26files-3D-26repos-<wbr>3D&d=DwIGaQ&c=LFYZ-o9_<wbr>HUMeMTSQicvjIg&r=X4GwEru-<wbr>SJ9DRnCxhze-aw&m=aN-<wbr>OTm6qpDxNIXC86mUeowDuZe9O-<wbr>NeCWHJdSvrVsYA&s=Lw4Nep7Zyf-<wbr>sUfbKo8DeOjVzVMWKxOwfD3G2MzZXj<wbr>AQ&e=</a><br>
>>> [4] <a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__review.openstack.org_-23_c_456843_&d=DwIGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=X4GwEru-SJ9DRnCxhze-aw&m=aN-OTm6qpDxNIXC86mUeowDuZe9O-NeCWHJdSvrVsYA&s=MkpWWcQzrowgdxFsD1P4Mn4XvEKvM2Ym8DMKcq5KLNg&e=" rel="noreferrer" target="_blank">https://urldefense.proofpoint.<wbr>com/v2/url?u=https-3A__review.<wbr>openstack.org_-23_c_456843_&d=<wbr>DwIGaQ&c=LFYZ-o9_<wbr>HUMeMTSQicvjIg&r=X4GwEru-<wbr>SJ9DRnCxhze-aw&m=aN-<wbr>OTm6qpDxNIXC86mUeowDuZe9O-<wbr>NeCWHJdSvrVsYA&s=<wbr>MkpWWcQzrowgdxFsD1P4Mn4XvEKvM2<wbr>Ym8DMKcq5KLNg&e=</a><br>
<span class="">>>><br>
>>><br>
>>> ______________________________<wbr>______________________________<wbr>______________<br>
>>> OpenStack Development Mailing List (not for usage questions)<br>
>>> Unsubscribe:<br>
>>> <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.<wbr>openstack.org?subject:<wbr>unsubscribe</a><br>
</span>>>> <a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openstack.org_cgi-2Dbin_mailman_listinfo_openstack-2Ddev&d=DwIGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=X4GwEru-SJ9DRnCxhze-aw&m=aN-OTm6qpDxNIXC86mUeowDuZe9O-NeCWHJdSvrVsYA&s=cjW0GYqZtCKw-P5ikZF_OPLkwsINaR4MckrDmIQ2drs&e=" rel="noreferrer" target="_blank">https://urldefense.proofpoint.<wbr>com/v2/url?u=http-3A__lists.<wbr>openstack.org_cgi-2Dbin_<wbr>mailman_listinfo_openstack-<wbr>2Ddev&d=DwIGaQ&c=LFYZ-o9_<wbr>HUMeMTSQicvjIg&r=X4GwEru-<wbr>SJ9DRnCxhze-aw&m=aN-<wbr>OTm6qpDxNIXC86mUeowDuZe9O-<wbr>NeCWHJdSvrVsYA&s=cjW0GYqZtCKw-<wbr>P5ikZF_<wbr>OPLkwsINaR4MckrDmIQ2drs&e=</a><br>
<span class="">>>><br>
>><br>
>><br>
>><br>
>> --<br>
>> Regards,<br>
>> Rabi Mishra<br>
>><br>
>> ______________________________<wbr>______________________________<wbr>______________<br>
>> OpenStack Development Mailing List (not for usage questions)<br>
>> Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.<wbr>openstack.org?subject:<wbr>unsubscribe</a><br>
</span>>> <a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openstack.org_cgi-2Dbin_mailman_listinfo_openstack-2Ddev&d=DwIGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=X4GwEru-SJ9DRnCxhze-aw&m=aN-OTm6qpDxNIXC86mUeowDuZe9O-NeCWHJdSvrVsYA&s=cjW0GYqZtCKw-P5ikZF_OPLkwsINaR4MckrDmIQ2drs&e=" rel="noreferrer" target="_blank">https://urldefense.proofpoint.<wbr>com/v2/url?u=http-3A__lists.<wbr>openstack.org_cgi-2Dbin_<wbr>mailman_listinfo_openstack-<wbr>2Ddev&d=DwIGaQ&c=LFYZ-o9_<wbr>HUMeMTSQicvjIg&r=X4GwEru-<wbr>SJ9DRnCxhze-aw&m=aN-<wbr>OTm6qpDxNIXC86mUeowDuZe9O-<wbr>NeCWHJdSvrVsYA&s=cjW0GYqZtCKw-<wbr>P5ikZF_<wbr>OPLkwsINaR4MckrDmIQ2drs&e=</a><br>
<span class="">><br>
><br>
> ______________________________<wbr>______________________________<wbr>______________<br>
> OpenStack Development Mailing List (not for usage questions)<br>
> Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.<wbr>openstack.org?subject:<wbr>unsubscribe</a><br>
</span>> <a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openstack.org_cgi-2Dbin_mailman_listinfo_openstack-2Ddev&d=DwIGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=X4GwEru-SJ9DRnCxhze-aw&m=aN-OTm6qpDxNIXC86mUeowDuZe9O-NeCWHJdSvrVsYA&s=cjW0GYqZtCKw-P5ikZF_OPLkwsINaR4MckrDmIQ2drs&e=" rel="noreferrer" target="_blank">https://urldefense.proofpoint.<wbr>com/v2/url?u=http-3A__lists.<wbr>openstack.org_cgi-2Dbin_<wbr>mailman_listinfo_openstack-<wbr>2Ddev&d=DwIGaQ&c=LFYZ-o9_<wbr>HUMeMTSQicvjIg&r=X4GwEru-<wbr>SJ9DRnCxhze-aw&m=aN-<wbr>OTm6qpDxNIXC86mUeowDuZe9O-<wbr>NeCWHJdSvrVsYA&s=cjW0GYqZtCKw-<wbr>P5ikZF_<wbr>OPLkwsINaR4MckrDmIQ2drs&e=</a><br>
<span class="">><br>
<br>
______________________________<wbr>______________________________<wbr>______________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.<wbr>openstack.org?subject:<wbr>unsubscribe</a><br>
</span><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openstack.org_cgi-2Dbin_mailman_listinfo_openstack-2Ddev&d=DwIGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=X4GwEru-SJ9DRnCxhze-aw&m=aN-OTm6qpDxNIXC86mUeowDuZe9O-NeCWHJdSvrVsYA&s=cjW0GYqZtCKw-P5ikZF_OPLkwsINaR4MckrDmIQ2drs&e=" rel="noreferrer" target="_blank">https://urldefense.proofpoint.<wbr>com/v2/url?u=http-3A__lists.<wbr>openstack.org_cgi-2Dbin_<wbr>mailman_listinfo_openstack-<wbr>2Ddev&d=DwIGaQ&c=LFYZ-o9_<wbr>HUMeMTSQicvjIg&r=X4GwEru-<wbr>SJ9DRnCxhze-aw&m=aN-<wbr>OTm6qpDxNIXC86mUeowDuZe9O-<wbr>NeCWHJdSvrVsYA&s=cjW0GYqZtCKw-<wbr>P5ikZF_<wbr>OPLkwsINaR4MckrDmIQ2drs&e=</a><br>
<div class="HOEnZb"><div class="h5">______________________________<wbr>______________________________<wbr>______________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.<wbr>openstack.org?subject:<wbr>unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack-dev</a><br>
</div></div></blockquote></div><br></div></div>