<html><body><p>Thanks Eli for the analysis. I notice that the time to download the image is only around 1:15 mins out of some 21 mins to set up devstack. So it seems trying to reduce the size of the image won't make a significant improvement in the devstack time. I wonder how the image size affects the VM creation time for the cluster. If we can look at the Heat event stream, we might get an idea. <br>Ton,<br><br><br><img width="16" height="16" src="cid:1__=07BBF46FDF897A9D8f9e8a93df938690918c07B@" border="0" alt="Inactive hide details for Egor Guz ---11/12/2015 05:25:15 PM---Eli, First of all I would like to say thank you for your effort "><font color="#424282">Egor Guz ---11/12/2015 05:25:15 PM---Eli, First of all I would like to say thank you for your effort (I never seen so many path sets ;)),</font><br><br><font size="2" color="#5F5F5F">From: </font><font size="2">Egor Guz <EGuz@walmartlabs.com></font><br><font size="2" color="#5F5F5F">To: </font><font size="2">"openstack-dev@lists.openstack.org" <openstack-dev@lists.openstack.org></font><br><font size="2" color="#5F5F5F">Date: </font><font size="2">11/12/2015 05:25 PM</font><br><font size="2" color="#5F5F5F">Subject: </font><font size="2">Re: [openstack-dev] [Magnum][Testing] Reduce Functional testing on gate.</font><br><hr width="100%" size="2" align="left" noshade style="color:#8091A5; "><br><br><br><tt>Eli,<br><br>First of all I would like to say thank you for your effort (I never seen so many path sets ;)), but I don’t think we should remove “tls_disabled=True” tests from gates now (maybe in L).<br>It’s still vey commonly used feature and backup plan if TLS doesn’t work for some reasons.<br><br>I think it’s good idea to group tests per pipeline we should definitely follow it.<br><br>—<br>Egor<br><br>From: "Qiao,Liyong" <liyong.qiao@intel.com<</tt><tt><a href="mailto:liyong.qiao@intel.com">mailto:liyong.qiao@intel.com</a></tt><tt>>><br>Reply-To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org<</tt><tt><a href="mailto:openstack-dev@lists.openstack.org">mailto:openstack-dev@lists.openstack.org</a></tt><tt>>><br>Date: Wednesday, November 11, 2015 at 23:02<br>To: "openstack-dev@lists.openstack.org<</tt><tt><a href="mailto:openstack-dev@lists.openstack.org">mailto:openstack-dev@lists.openstack.org</a></tt><tt>>" <openstack-dev@lists.openstack.org<</tt><tt><a href="mailto:openstack-dev@lists.openstack.org">mailto:openstack-dev@lists.openstack.org</a></tt><tt>>><br>Subject: [openstack-dev] [Magnum][Testing] Reduce Functional testing on gate.<br><br>hello all:<br><br>I will update some Magnum functional testing status, functional/integration testing<br>is important to us, since we change/modify the Heat template rapidly, we need to<br>verify the modification is correct, so we need to cover all templates Magnum has.<br>and currently we only has k8s testing(only test with atomic image), we need to<br>add more, like swarm(WIP), mesos(under plan), also , we may need to support COS image.<br>lots of work need to be done.<br><br>for the functional testing time costing, we discussed during the Tokyo summit,<br>Adrian expected that we can reduce the timing cost to 20min.<br><br>I did some analyses on the functional/integrated testing on gate pipeline.<br>the stages will be follows:<br>take k8s functional testing for example, we did follow testing case:<br><br>1) baymodel creation<br>2) bay(tls_disabled=True) creation/deletion<br>3) bay(tls_disabled=False) creation to testing k8s api and delete it after testing.<br><br>for each stage, the time costing is follows:<br><br> * devstack prepare: 5-6 mins<br> * Running devstack: 15 mins(include downloading atomic image)<br> * 1) and 2) 15 mins<br> * 3) 15 +3 mins<br><br>totally about 60 mins currently a example is 1h 05m 57s<br>see </tt><tt><a href="http://logs.openstack.org/10/243910/1/check/gate-functional-dsvm-magnum-k8s/5e61039/console.html">http://logs.openstack.org/10/243910/1/check/gate-functional-dsvm-magnum-k8s/5e61039/console.html</a></tt><tt><br>for all time stamps.<br><br>I don't think it is possible to reduce time to 20 mins, since devstack setup will take 20 mins already.<br><br>To reduce time, I suggest to only create 1 bay each pipeline and do vary kinds of testing<br>on this bay, if want to test some specify bay (for example, network_driver etc), create<br>a new pipeline .<br><br>So, Ｉ think we can delete 2), since 3) will do similar things(create/delete), the different is<br>3) use tls_disabled=False. what do you think ?<br>see </tt><tt><a href="https://review.openstack.org/244378">https://review.openstack.org/244378</a></tt><tt> for the time costing, will reduce to 45 min (48m 50s in the example.)<br><br>=================================<br>For other related functional testing works:<br>I 'v done the split of functional testing per COE, we have pipeline as:<br><br> * gate-functional-dsvm-magnum-api 30 mins<br> * gate-functional-dsvm-magnum-k8s 60 mins<br><br>And for swam pipeline, patches is done, under reviewing now(works fine on gate)<br></tt><tt><a href="https://review.openstack.org/244391">https://review.openstack.org/244391</a></tt><tt><br></tt><tt><a href="https://review.openstack.org/226125">https://review.openstack.org/226125</a></tt><tt><br><br>========================<br><br>--<br>BR, Eli(Li Yong)Qiao<br>__________________________________________________________________________<br>OpenStack Development Mailing List (not for usage questions)<br>Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe<br></tt><tt><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a></tt><tt><br></tt><br><BR>
</body></html>