Open Stack

On Mon, Feb 15, 2016, at 11:48 AM, Ivan Kolodyazhny wrote:
> Hi all,
> 
> I'll talk mostly about python-cinderclient but the same question could be
> related for other clients.
> 
> Now, for python-cinderclient we've got to kinds for functional/integrated
> jobs:
> 
> 1) gate-cinderclient-dsvm-functional - a very limited (for now) set of
> functional tests, most of them were part of tempest CLI tests in the
> past.
> 
> 2) gate-tempest-dsvm-neutron-src-python-cinderclient - if I understand
> right, the idea os this job was to have integrated tests to test
> cinderclient with other projects to verify that new patch to
> python-cinderclietn won't break any other project.
> But it does *not* test cinderclient at all, except few attach-related
> tests
> because Tempest doesn't use python-*client.

Tempest doesn't use python-*client to talk to the APIs but the various
OpenStack services do use python-*client to talk to the other services.
Using cinderclient as an example, nova consumes cinderclient to perform
volume operations in nova/volume/cinder.py. There is value in this
existing test if those code paths are exercised. Basically ensuring the
next release of cinderclient does not break nova. It may be the case
that cinderclient is a bad example because tempest doesn't do volume
operations through nova, but I am sure for many of the other clients
these tests do provide value.

> 
> The same job was added for python-heatclient but was removed because
> devstack didn't install Heat for that job [1].
> 
> We agreed [2] to remove this job from cinderclient gates too, once
> functional or integration tests will be implemented.

Just make sure that you don't lose exercising of the above code paths
when this transition happens. If we don't currently test that code it
would be a good goal for any new integration testing to do so.

> 
> 
> There is a proposal to python-cinderclient tests to implement some
> cross-project testing to make sure, that new python-cinderclient won't
> break any of existing project who use it.
> 
> After discussing in IRC with John Griffith (jgriffith) I'm realized that
> it
> could be an cross-project initiative in such kind of integration tests.
> OpenStack Client (OSC) could cover some part of such tests, but does it
> mean that we'll run OSC tests on every patch to python-*client? We can
> run
> only cinder-realated OSC tests on our gates to verify that it doesn't
> breack OSC and, may be other project.
> 
> The other option, is to implement tests like [3] per project basis and
> call
> it "integration".  Such tests could cover more cases than OSC functional
> tests and have more project-related test cases, e.g.: test some
> python-cinderclient specific corner cases, which is not related to OSC.
> 
> IMO, It would be good to have some cross-project decision on how will be
> implement clients' integration tests per project.
> 
> 
> [1] https://review.openstack.org/#/c/272411/
> [2]
> http://eavesdrop.openstack.org/meetings/cinder/2015/cinder.2015-12-16-16.00.log.html
> [3] https://review.openstack.org/#/c/279432/8
> 
> Regards,
> Ivan Kolodyazhny,
> http://blog.e0ne.info/