[openstack-dev] [cinder][all] Integration python-*client tests on gates
e0ne at e0ne.info
Wed Mar 2 10:24:57 UTC 2016
I do understand why we have tempest for python-cinderclient now.
But my point is: tempest runs more than 200 tests per each cinderclient
change request which takes a lot of time. Why can't we just introduce few
integration tests which will tests nova<=>python-cinderclient API.
Also, Nova is not only one consumer of cinderclient. What about Heat? We
don't want to break it too but to run all Heat-related Tempest tests is not
a good idea. We have to implement integration tests between Heat and
On Tue, Feb 16, 2016 at 2:05 PM, Sean Dague <sean at dague.net> wrote:
> On 02/15/2016 02:48 PM, Ivan Kolodyazhny wrote:
> > Hi all,
> > I'll talk mostly about python-cinderclient but the same question could
> > be related for other clients.
> > Now, for python-cinderclient we've got to kinds for
> > functional/integrated jobs:
> > 1) gate-cinderclient-dsvm-functional - a very limited (for now) set of
> > functional tests, most of them were part of tempest CLI tests in the
> > 2) gate-tempest-dsvm-neutron-src-python-cinderclient - if I understand
> > right, the idea os this job was to have integrated tests to test
> > cinderclient with other projects to verify that new patch to
> > python-cinderclietn won't break any other project.
> > But it does *not* test cinderclient at all, except few attach-related
> > tests because Tempest doesn't use python-*client.
> This does test the real world usage of Nova consuming
> python-cinderclient. That's why it's still there. This ensures that a
> cinderclient upcoming release won't completely tank the integrated gate.
> All openstack libraries that get used by all servers in openstack have
> something equivalent.
> > The same job was added for python-heatclient but was removed because
> > devstack didn't install Heat for that job .
> > We agreed  to remove this job from cinderclient gates too, once
> > functional or integration tests will be implemented.
> Um, what now?
> > There is a proposal to python-cinderclient tests to implement some
> > cross-project testing to make sure, that new python-cinderclient won't
> > break any of existing project who use it.
> > After discussing in IRC with John Griffith (jgriffith) I'm realized that
> > it could be an cross-project initiative in such kind of integration
> > tests. OpenStack Client (OSC) could cover some part of such tests, but
> > does it mean that we'll run OSC tests on every patch to python-*client?
> > We can run only cinder-realated OSC tests on our gates to verify that it
> > doesn't breack OSC and, may be other project.
> > The other option, is to implement tests like  per project basis and
> > call it "integration". Such tests could cover more cases than OSC
> > functional tests and have more project-related test cases, e.g.: test
> > some python-cinderclient specific corner cases, which is not related to
> > IMO, It would be good to have some cross-project decision on how will be
> > implement clients' integration tests per project.
> >  https://review.openstack.org/#/c/272411/
> > 
> >  https://review.openstack.org/#/c/279432/8
> > Regards,
> > Ivan Kolodyazhny,
> > http://blog.e0ne.info/
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> Sean Dague
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the OpenStack-dev