<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 07/03/14 01:53, Sean Dague wrote:<br>
</div>
<blockquote cite="mid:53186FAF.5080308@dague.net" type="cite">
<pre wrap="">We're at Freeze, so I want to pick up and understand where we currently
stand with both Neutron and Heat actually getting tested fully in the gate.
First Neutron -
<a class="moz-txt-link-freetext" href="https://blueprints.launchpad.net/tempest/+spec/fix-gate-tempest-devstack-vm-quantum-full">https://blueprints.launchpad.net/tempest/+spec/fix-gate-tempest-devstack-vm-quantum-full</a>
We know that this is *close* as the full job is running non voting
everywhere, and typically passing. How close are we? Or should we be
defering this until Juno (which would be unfortunate).
Second Heat -
<a class="moz-txt-link-freetext" href="https://blueprints.launchpad.net/tempest/+spec/tempest-heat-integration">https://blueprints.launchpad.net/tempest/+spec/tempest-heat-integration</a>
The Heat tests that are in a normal Tempest job are relatively trivial
surface verification, and in no way actually make sure that Heat is
operating at a real level. This fact is a contributing factor to why
Heat was broken in i2.
The first real verification for Heat is in the Heat slow job (which we
created to give Heat a separate time budget, because doing work that
requires real guests takes time).
The heat slow job looks like it is finally passing much of the time -
<a class="moz-txt-link-freetext" href="http://logstash.openstack.org/#eyJzZWFyY2giOiIobWVzc2FnZTpcIkZpbmlzaGVkOiBTVUNDRVNTXCIgT1IgbWVzc2FnZTpcIkZpbmlzaGVkOiBGQUlMVVJFXCIpIEFORCBidWlsZF9uYW1lOmNoZWNrLXRlbXBlc3QtZHN2bS1uZXV0cm9uLWhlYXQtc2xvdyIsImZpZWxkcyI6W10sIm9mZnNldCI6MCwidGltZWZyYW1lIjoiNjA0ODAwIiwiZ3JhcGhtb2RlIjoiY291bnQiLCJ0aW1lIjp7InVzZXJfaW50ZXJ2YWwiOjB9LCJzdGFtcCI6MTM5NDEwOTg3NDQ4OH0=">http://logstash.openstack.org/#eyJzZWFyY2giOiIobWVzc2FnZTpcIkZpbmlzaGVkOiBTVUNDRVNTXCIgT1IgbWVzc2FnZTpcIkZpbmlzaGVkOiBGQUlMVVJFXCIpIEFORCBidWlsZF9uYW1lOmNoZWNrLXRlbXBlc3QtZHN2bS1uZXV0cm9uLWhlYXQtc2xvdyIsImZpZWxkcyI6W10sIm9mZnNldCI6MCwidGltZWZyYW1lIjoiNjA0ODAwIiwiZ3JhcGhtb2RlIjoiY291bnQiLCJ0aW1lIjp7InVzZXJfaW50ZXJ2YWwiOjB9LCJzdGFtcCI6MTM5NDEwOTg3NDQ4OH0=</a>
It's seeing a 78% pass rate in check. Can anyone in the Heat team
confirm that the Failures in this job are actually real failures on
patches that should have been blocked?
I'd like to get that turned on (and on all the projects) as soon as the
Heat team is confident on it so that Heat actually participates in the
tempest/devstack gate in a material way and we can prevent future issues
where a keystone, nova, neutron or whatever change would break Heat in git.
</pre>
</blockquote>
I've raised <a class="moz-txt-link-freetext" href="https://bugs.launchpad.net/tempest/+bug/1288970">https://bugs.launchpad.net/tempest/+bug/1288970</a> to track
the most common error (
<meta http-equiv="content-type" content="text/html;
charset=ISO-8859-1">
<span style="color: rgb(51, 51, 51); font-family: 'Ubuntu Mono',
monospace; font-size: 11.818181991577148px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing: normal;
line-height: 18px; orphans: auto; text-align: left; text-indent:
0px; text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px;
background-color: rgb(255, 255, 255); display: inline !important;
float: none;">NeutronResource</span><span style="color: rgb(51,
51, 51); font-family: 'Ubuntu Mono', monospace; font-size:
11.818181991577148px; font-style: normal; font-variant: normal;
font-weight: normal; letter-spacing: normal; line-height: 18px;
orphans: auto; text-align: left; text-indent: 0px; text-transform:
none; white-space: normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px; background-color: rgb(255, 255,
255); display: inline !important; float: none;">sTestJSON f</span>
<meta http-equiv="content-type" content="text/html;
charset=ISO-8859-1">
<span style="color: rgb(51, 51, 51); font-family: 'Ubuntu Mono',
monospace; font-size: 11.818181991577148px; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing: normal;
line-height: 18px; orphans: auto; text-align: left; text-indent:
0px; text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px;
background-color: rgb(255, 255, 255); display: inline !important;
float: none;">ailed to reach CREATE_COMPLETE status within the
required time (300 s)</span>)<br>
<br>
tl;dr sometimes booting alone is taking 244s, so I think this test
would be significantly more reliable if the timeout was raised.<br>
<br>
I would propose raising the default orchestration build_timeout to
600s for now, but it may need to go up to 1200s when the autoscaling
scenario is enabled again.<br>
<br>
<a class="moz-txt-link-freetext" href="https://review.openstack.org/#/c/78756/">https://review.openstack.org/#/c/78756/</a><br>
<br>
Once this change is in heat-slow should be reliable enough to make
it voting.<br>
</body>
</html>