<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 12/07/14 09:25, Joe Gordon wrote:<br>
</div>
<blockquote
cite="mid:CAHXdxOfcnnO-0f3mnPtRQURCDZrghZ7R8TeJMm6_KRD9tQY6YA@mail.gmail.com"
type="cite">
<div dir="ltr"><br>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Fri, Jul 11, 2014 at 4:42 AM,
Jeremy Stanley <span dir="ltr"><<a
moz-do-not-send="true" href="mailto:fungi@yuggoth.org"
target="_blank">fungi@yuggoth.org</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div class="">On 2014-07-11 11:21:19 +0200 (+0200),
Matthias Runge wrote:<br>
> this broke horizon stable and master; heat stable
is affected as<br>
> well.<br>
</div>
[...]<br>
<br>
I guess this is a plea for applying something like the
oslotest<br>
framework to client libraries so they get backward-compat
jobs run<br>
against unit tests of all dependant/consuming software...
branchless<br>
tempest already alleviates some of this, but not the case
of changes<br>
in a library which will break unit/functional tests of
another<br>
project.<br>
</blockquote>
<div><br>
</div>
<div>We actually do have some tests for backwards
compatibility, and they all passed. Presumably because
both heat and horizon have poor integration test.</div>
<div><br>
</div>
<div>
We ran </div>
<div><br>
</div>
<div>
<ul class=""
style="color:rgb(0,0,0);font-family:sans-serif">
<li class="" style="list-style-type:none"><span class=""
style="display:inline-block;min-width:20em;padding-bottom:2pt;width:auto!important"><a
moz-do-not-send="true"
href="http://logs.openstack.org/66/94166/3/check/check-tempest-dsvm-full-havana/8e09faa"
style="text-decoration:none;color:rgb(6,84,172)">check-tempest-dsvm-full-havana</a></span> <span
class=""><span class="" style="color:rgb(0,127,0)">SUCCESS</span> in
40m 47s (non-voting)</span></li>
<li class="" style="list-style-type:none"><span class=""
style="display:inline-block;min-width:20em;padding-bottom:2pt;width:auto!important"><a
moz-do-not-send="true"
href="http://logs.openstack.org/66/94166/3/check/check-tempest-dsvm-neutron-havana/b4ad019"
style="text-decoration:none;color:rgb(6,84,172)">check-tempest-dsvm-neutron-havana</a></span> <span
class=""><span class="" style="color:rgb(0,127,0)">SUCCESS</span> in
36m 17s (non-voting)</span></li>
<li class="" style="list-style-type:none"><span class=""
style="display:inline-block;min-width:20em;padding-bottom:2pt;width:auto!important"><a
moz-do-not-send="true"
href="http://logs.openstack.org/66/94166/3/check/check-tempest-dsvm-full-icehouse/c0c62e5"
style="text-decoration:none;color:rgb(6,84,172)">check-tempest-dsvm-full-icehouse</a></span> <span
class=""><span class="" style="color:rgb(0,127,0)">SUCCESS</span> in
53m 05s</span></li>
<li class="" style="list-style-type:none"><span class=""
style="display:inline-block;min-width:20em;padding-bottom:2pt;width:auto!important"><a
moz-do-not-send="true"
href="http://logs.openstack.org/66/94166/3/check/check-tempest-dsvm-neutron-icehouse/a54aedb"
style="color:rgb(6,84,172)">check-tempest-dsvm-neutron-icehouse</a></span> <span
class=""><span class="" style="color:rgb(0,127,0)">SUCCESS</span> in
57m 28s</span></li>
</ul>
</div>
<div><br>
</div>
<div>on the offending patches (<a moz-do-not-send="true"
href="https://review.openstack.org/#/c/94166/">https://review.openstack.org/#/c/94166/</a>)</div>
<div> </div>
<div><br>
</div>
<div>Infra patch that added these tests: <a
moz-do-not-send="true"
href="https://review.openstack.org/#/c/80698/">https://review.openstack.org/#/c/80698/</a><br>
</div>
<div><br>
</div>
<br>
</div>
</div>
</div>
</blockquote>
Heat-proper would have continued working fine with novaclient
2.18.0. The regression was with raising novaclient exceptions, which
is only required in our unit tests. I saw this break coming and
switched to raising via from_response
<a class="moz-txt-link-freetext" href="https://review.openstack.org/#/c/97977/22/heat/tests/v1_1/fakes.py">https://review.openstack.org/#/c/97977/22/heat/tests/v1_1/fakes.py</a><br>
<br>
Unit tests tend to deal with more internals of client libraries just
for mocking purposes, and there have been multiple breaks in unit
tests for heat and horizon when client libraries make internal
changes.<br>
<br>
This could be avoided if the client gate jobs run the unit tests for
the projects which consume them.<br>
</body>
</html>