Open Stack

Eric,

For now, we test Cinder API with some concurrency only with Rally, so, IMO,
it's reasonable get more scenarios for API races fixes.

It's not a hard task to implement new scenarios, they are pretty simple:
[11] and [12]

[11]
https://github.com/openstack/rally/blob/master/rally/plugins/openstack/scenarios/cinder/volumes.py#L535
[12]
https://github.com/openstack/rally/blob/master/rally-jobs/cinder.yaml#L516

Regards,
Ivan Kolodyazhny,
http://blog.e0ne.info/

On Wed, Mar 2, 2016 at 4:50 PM, Eric Harney <eharney at redhat.com> wrote:

> On 03/02/2016 09:36 AM, Ivan Kolodyazhny wrote:
> > Eric,
> >
> > There are Gorka's patches [10] to remove API Races
> >
> >
> > [10]
> >
> https://review.openstack.org/#/q/project:openstack/cinder+branch:master+topic:fix/api-races-simplified
> >
> > Regards,
> > Ivan Kolodyazhny,
> > http://blog.e0ne.info/
> >
>
> So the second part of my question is, is writing a Rally job to prove
> out that code a reasonable task?
>
> How hard is that to do and what does it look like?
>
> > On Wed, Mar 2, 2016 at 4:27 PM, Eric Harney <eharney at redhat.com> wrote:
> >
> >> On 03/02/2016 06:25 AM, Ivan Kolodyazhny wrote:
> >>> Hi Team,
> >>>
> >>> Here are my thoughts and proposals how to make Cinder testing process
> >>> better. I won't cover "3rd party CI's" topic here. I will share my
> >> opinion
> >>> about current and feature jobs.
> >>>
> >>>
> >>> Unit-tests
> >>>
> >>>    - Long-running tests. I hope, everybody will agree that unit-tests
> >> must
> >>>    be quite simple and very fast. Unit tests which takes more than 3-5
> >> seconds
> >>>    should be refactored and/or moved to 'integration' tests.
> >>>    Thanks to Tom Barron for several fixes like [1]. IMO, we it would be
> >>>    good to have some hacking checks to prevent such issues in a future.
> >>>
> >>>    - Tests coverage. We don't check it in an automatic way on gates.
> >>>    Usually, we require to add some unit-tests during code review
> >> process. Why
> >>>    can't we add coverage job to our CI and do not merge new patches,
> with
> >>>    will decrease tests coverage rate? Maybe, such job could be voting
> in
> >> a
> >>>    future to not ignore it. For now, there is not simple way to check
> >> coverage
> >>>    because 'tox -e cover' output is not useful [2].
> >>>
> >>>
> >>> Functional tests for Cinder
> >>>
> >>> We introduced some functional tests last month [3]. Here is a patch to
> >>> infra to add new job [4]. Because these tests were moved from
> >> unit-tests, I
> >>> think we're OK to make this job voting. Such tests should not be a
> >>> replacement for Tempest. They even could tests Cinder with Fake Driver
> to
> >>> make it faster and not related on storage backends issues.
> >>>
> >>>
> >>> Tempest in-tree tests
> >>>
> >>> Sean started work on it [5] and I think it's a good idea to get them in
> >>> Cinder repo to run them on Tempest jobs and 3-rd party CIs against a
> real
> >>> backend.
> >>>
> >>>
> >>> Functional tests for python-brick-cinderclient-ext
> >>>
> >>> There are patches that introduces functional tests [6] and new job [7].
> >>>
> >>>
> >>> Functional tests for python-cinderclient
> >>>
> >>> We've got a very limited set of such tests and non-voting job. IMO, we
> >> can
> >>> run them even with Cinder Fake Driver to make them not depended on a
> >>> storage backend and make it faster. I believe, we can make this job
> >> voting
> >>> soon. Also, we need more contributors to this kind of tests.
> >>>
> >>>
> >>> Integrated tests for python-cinderclient
> >>>
> >>> We need such tests to make sure that we won't break Nova, Heat or other
> >>> python-cinderclient consumers with a next merged patch. There is a
> thread
> >>> in openstack-dev ML about such tests [8] and proposal [9] to introduce
> >> them
> >>> to python-cinderclient.
> >>>
> >>>
> >>> Rally tests
> >>>
> >>> IMO, it would be good to have new Rally scenarios for every patches
> like
> >>> 'improves performance', 'fixes concurrency issues', etc. Even if we as
> a
> >>> Cinder community don't have enough time to implement them, we have to
> ask
> >>> for them in reviews, openstack-dev ML, file Rally bugs and blueprints
> if
> >>> needed.
> >>>
> >>
> >> Are there any recent examples of a fix like this recently where it would
> >> seem like a reasonable task to write a Rally scenario along with the
> patch?
> >>
> >> Not being very familiar with Rally (as I think most of us aren't), I'm
> >> having a hard time picturing this.
> >>
> >>>
> >>> [1] https://review.openstack.org/#/c/282861/
> >>> [2] http://paste.openstack.org/show/488925/
> >>> [3] https://review.openstack.org/#/c/267801/
> >>> [4] https://review.openstack.org/#/c/287115/
> >>> [5] https://review.openstack.org/#/c/274471/
> >>> [6] https://review.openstack.org/#/c/265811/
> >>> [7] https://review.openstack.org/#/c/265925/
> >>> [8]
> >>>
> >>
> http://lists.openstack.org/pipermail/openstack-dev/2016-March/088027.html
> >>> [9] https://review.openstack.org/#/c/279432/
> >>>
> >>>
> >>> Regards,
> >>> Ivan Kolodyazhny,
> >>> http://blog.e0ne.info/
> >>>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160302/06532999/attachment.html>