Open Stack

On Wed, Jul 16, 2014 at 11:28 PM, Steve Baker <sbaker at redhat.com> wrote:

>  On 12/07/14 09:25, Joe Gordon wrote:
>
>
>
>
> On Fri, Jul 11, 2014 at 4:42 AM, Jeremy Stanley <fungi at yuggoth.org> wrote:
>
>> On 2014-07-11 11:21:19 +0200 (+0200), Matthias Runge wrote:
>> > this broke horizon stable and master; heat stable is affected as
>> > well.
>>  [...]
>>
>> I guess this is a plea for applying something like the oslotest
>> framework to client libraries so they get backward-compat jobs run
>> against unit tests of all dependant/consuming software... branchless
>> tempest already alleviates some of this, but not the case of changes
>> in a library which will break unit/functional tests of another
>> project.
>>
>
>  We actually do have some tests for backwards compatibility, and they all
> passed. Presumably because both heat and horizon have poor integration test.
>
>  We ran
>
>
>    - check-tempest-dsvm-full-havana
>    <http://logs.openstack.org/66/94166/3/check/check-tempest-dsvm-full-havana/8e09faa>
>     SUCCESS in 40m 47s (non-voting)
>    - check-tempest-dsvm-neutron-havana
>    <http://logs.openstack.org/66/94166/3/check/check-tempest-dsvm-neutron-havana/b4ad019>
>     SUCCESS in 36m 17s (non-voting)
>    - check-tempest-dsvm-full-icehouse
>    <http://logs.openstack.org/66/94166/3/check/check-tempest-dsvm-full-icehouse/c0c62e5>
>     SUCCESS in 53m 05s
>    - check-tempest-dsvm-neutron-icehouse
>    <http://logs.openstack.org/66/94166/3/check/check-tempest-dsvm-neutron-icehouse/a54aedb>
>     SUCCESS in 57m 28s
>
>
>  on the offending patches (https://review.openstack.org/#/c/94166/)
>
>
>  Infra patch that added these tests:
> https://review.openstack.org/#/c/80698/
>
>
>   Heat-proper would have continued working fine with novaclient 2.18.0.
> The regression was with raising novaclient exceptions, which is only
> required in our unit tests. I saw this break coming and switched to raising
> via from_response
> https://review.openstack.org/#/c/97977/22/heat/tests/v1_1/fakes.py
>
> Unit tests tend to deal with more internals of client libraries just for
> mocking purposes, and there have been multiple breaks in unit tests for
> heat and horizon when client libraries make internal changes.
>
> This could be avoided if the client gate jobs run the unit tests for the
> projects which consume them.
>

That may work but isn't this exactly what integration testing is for?


> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140717/259d8b2d/attachment.html>