[openstack-dev] [cinder] Proposal: changes to our current testing process
e0ne at e0ne.info
Wed Mar 2 14:36:19 UTC 2016
There are Gorka's patches  to remove API Races
On Wed, Mar 2, 2016 at 4:27 PM, Eric Harney <eharney at redhat.com> wrote:
> On 03/02/2016 06:25 AM, Ivan Kolodyazhny wrote:
> > Hi Team,
> > Here are my thoughts and proposals how to make Cinder testing process
> > better. I won't cover "3rd party CI's" topic here. I will share my
> > about current and feature jobs.
> > Unit-tests
> > - Long-running tests. I hope, everybody will agree that unit-tests
> > be quite simple and very fast. Unit tests which takes more than 3-5
> > should be refactored and/or moved to 'integration' tests.
> > Thanks to Tom Barron for several fixes like . IMO, we it would be
> > good to have some hacking checks to prevent such issues in a future.
> > - Tests coverage. We don't check it in an automatic way on gates.
> > Usually, we require to add some unit-tests during code review
> process. Why
> > can't we add coverage job to our CI and do not merge new patches, with
> > will decrease tests coverage rate? Maybe, such job could be voting in
> > future to not ignore it. For now, there is not simple way to check
> > because 'tox -e cover' output is not useful .
> > Functional tests for Cinder
> > We introduced some functional tests last month . Here is a patch to
> > infra to add new job . Because these tests were moved from
> unit-tests, I
> > think we're OK to make this job voting. Such tests should not be a
> > replacement for Tempest. They even could tests Cinder with Fake Driver to
> > make it faster and not related on storage backends issues.
> > Tempest in-tree tests
> > Sean started work on it  and I think it's a good idea to get them in
> > Cinder repo to run them on Tempest jobs and 3-rd party CIs against a real
> > backend.
> > Functional tests for python-brick-cinderclient-ext
> > There are patches that introduces functional tests  and new job .
> > Functional tests for python-cinderclient
> > We've got a very limited set of such tests and non-voting job. IMO, we
> > run them even with Cinder Fake Driver to make them not depended on a
> > storage backend and make it faster. I believe, we can make this job
> > soon. Also, we need more contributors to this kind of tests.
> > Integrated tests for python-cinderclient
> > We need such tests to make sure that we won't break Nova, Heat or other
> > python-cinderclient consumers with a next merged patch. There is a thread
> > in openstack-dev ML about such tests  and proposal  to introduce
> > to python-cinderclient.
> > Rally tests
> > IMO, it would be good to have new Rally scenarios for every patches like
> > 'improves performance', 'fixes concurrency issues', etc. Even if we as a
> > Cinder community don't have enough time to implement them, we have to ask
> > for them in reviews, openstack-dev ML, file Rally bugs and blueprints if
> > needed.
> Are there any recent examples of a fix like this recently where it would
> seem like a reasonable task to write a Rally scenario along with the patch?
> Not being very familiar with Rally (as I think most of us aren't), I'm
> having a hard time picturing this.
> >  https://review.openstack.org/#/c/282861/
> >  http://paste.openstack.org/show/488925/
> >  https://review.openstack.org/#/c/267801/
> >  https://review.openstack.org/#/c/287115/
> >  https://review.openstack.org/#/c/274471/
> >  https://review.openstack.org/#/c/265811/
> >  https://review.openstack.org/#/c/265925/
> > 
> >  https://review.openstack.org/#/c/279432/
> > Regards,
> > Ivan Kolodyazhny,
> > http://blog.e0ne.info/
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the OpenStack-dev