<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
</head>
<body bgcolor="#FFFFFF" text="#000000">
hello all:<br>
<br>
I will update some Magnum functional testing status,
functional/integration testing<br>
is important to us, since we change/modify the Heat template
rapidly, we need to<br>
verify the modification is correct, so we need to cover all
templates Magnum has.<br>
and currently we only has k8s testing(only test with atomic image),
we need to<br>
add more, like swarm(WIP), mesos(under plan), also , we may need to
support COS image.<br>
lots of work need to be done.<br>
<br>
for the functional testing time costing, we discussed during the
Tokyo summit,<br>
Adrian expected that we can reduce the timing cost to 20min.<br>
<br>
I did some analyses on the functional/integrated testing on gate
pipeline.<br>
the stages will be follows:<br>
take k8s functional testing for example, we did follow testing case:<br>
<br>
1) baymodel creation<br>
2) bay(tls_disabled=True) creation/deletion<br>
3) bay(tls_disabled=False) creation to testing k8s api and delete it
after testing.<br>
<br>
for each stage, the time costing is follows:<br>
<ul>
<li>devstack prepare: 5-6 mins</li>
<li>Running devstack: 15 mins(include downloading atomic image)</li>
<li>1) and 2) 15 mins</li>
<li>3) 15 +3 mins</li>
</ul>
totally about 60 mins currently a example is 1h 05m 57s<br>
see
<a class="moz-txt-link-freetext" href="http://logs.openstack.org/10/243910/1/check/gate-functional-dsvm-magnum-k8s/5e61039/console.html">http://logs.openstack.org/10/243910/1/check/gate-functional-dsvm-magnum-k8s/5e61039/console.html</a><br>
for all time stamps.<br>
<br>
I don't think it is possible to reduce time to 20 mins, since
devstack setup will take 20 mins already.<br>
<br>
To reduce time, I suggest to only create 1 bay each pipeline and do
vary kinds of testing<br>
on this bay, if want to test some specify bay (for example,
network_driver etc), create<br>
a new pipeline .<br>
<br>
So, Ｉ think we can <b>delete 2)</b>, since 3) will do similar
things(create/delete), the different is <br>
3) use tls_disabled=False. <b>what do you think </b>?<br>
see <a class="moz-txt-link-freetext" href="https://review.openstack.org/244378">https://review.openstack.org/244378</a> for the time costing, will
reduce to 45 min (48m 50s in the example.)<br>
<br>
=================================<br>
For other related functional testing works:<br>
I 'v done the split of functional testing per COE, we have pipeline
as:<br>
<ul>
<li>gate-functional-dsvm-magnum-api 30 mins</li>
<li>gate-functional-dsvm-magnum-k8s 60 mins</li>
</ul>
And for swam pipeline, patches is done, under reviewing now(works
fine on gate)<br>
<a class="moz-txt-link-freetext" href="https://review.openstack.org/244391">https://review.openstack.org/244391</a><br>
<a class="moz-txt-link-freetext" href="https://review.openstack.org/226125">https://review.openstack.org/226125</a><br>
<br>
========================<br>
<pre class="moz-signature" cols="72">--
BR, Eli(Li Yong)Qiao</pre>
</body>
</html>