Open Stack

I'll try to implement such scenario and step-by-step guideline soon.

Regards,
Ivan Kolodyazhny,
http://blog.e0ne.info/

On Wed, Mar 2, 2016 at 5:16 PM, Eric Harney <eharney at redhat.com> wrote:

> On 03/02/2016 10:07 AM, Ivan Kolodyazhny wrote:
> > Eric,
> >
> > For now, we test Cinder API with some concurrency only with Rally, so,
> IMO,
> > it's reasonable get more scenarios for API races fixes.
> >
> > It's not a hard task to implement new scenarios, they are pretty simple:
> > [11] and [12]
> >
>
> Sure, these are simple, but I think it's nowhere near that simple to
> write a scenario which will prove that "remove API races" works correctly.
>
> > [11]
> >
> https://github.com/openstack/rally/blob/master/rally/plugins/openstack/scenarios/cinder/volumes.py#L535
> > [12]
> >
> https://github.com/openstack/rally/blob/master/rally-jobs/cinder.yaml#L516
> >
> > Regards,
> > Ivan Kolodyazhny,
> > http://blog.e0ne.info/
> >
> > On Wed, Mar 2, 2016 at 4:50 PM, Eric Harney <eharney at redhat.com> wrote:
> >
> >> On 03/02/2016 09:36 AM, Ivan Kolodyazhny wrote:
> >>> Eric,
> >>>
> >>> There are Gorka's patches [10] to remove API Races
> >>>
> >>>
> >>> [10]
> >>>
> >>
> https://review.openstack.org/#/q/project:openstack/cinder+branch:master+topic:fix/api-races-simplified
> >>>
> >>> Regards,
> >>> Ivan Kolodyazhny,
> >>> http://blog.e0ne.info/
> >>>
> >>
> >> So the second part of my question is, is writing a Rally job to prove
> >> out that code a reasonable task?
> >>
> >> How hard is that to do and what does it look like?
> >>
> >>> On Wed, Mar 2, 2016 at 4:27 PM, Eric Harney <eharney at redhat.com>
> wrote:
> >>>
> >>>> On 03/02/2016 06:25 AM, Ivan Kolodyazhny wrote:
> >>>>> Hi Team,
> >>>>>
> >>>>> Here are my thoughts and proposals how to make Cinder testing process
> >>>>> better. I won't cover "3rd party CI's" topic here. I will share my
> >>>> opinion
> >>>>> about current and feature jobs.
> >>>>>
> >>>>>
> >>>>> Unit-tests
> >>>>>
> >>>>>    - Long-running tests. I hope, everybody will agree that unit-tests
> >>>> must
> >>>>>    be quite simple and very fast. Unit tests which takes more than
> 3-5
> >>>> seconds
> >>>>>    should be refactored and/or moved to 'integration' tests.
> >>>>>    Thanks to Tom Barron for several fixes like [1]. IMO, we it would
> be
> >>>>>    good to have some hacking checks to prevent such issues in a
> future.
> >>>>>
> >>>>>    - Tests coverage. We don't check it in an automatic way on gates.
> >>>>>    Usually, we require to add some unit-tests during code review
> >>>> process. Why
> >>>>>    can't we add coverage job to our CI and do not merge new patches,
> >> with
> >>>>>    will decrease tests coverage rate? Maybe, such job could be voting
> >> in
> >>>> a
> >>>>>    future to not ignore it. For now, there is not simple way to check
> >>>> coverage
> >>>>>    because 'tox -e cover' output is not useful [2].
> >>>>>
> >>>>>
> >>>>> Functional tests for Cinder
> >>>>>
> >>>>> We introduced some functional tests last month [3]. Here is a patch
> to
> >>>>> infra to add new job [4]. Because these tests were moved from
> >>>> unit-tests, I
> >>>>> think we're OK to make this job voting. Such tests should not be a
> >>>>> replacement for Tempest. They even could tests Cinder with Fake
> Driver
> >> to
> >>>>> make it faster and not related on storage backends issues.
> >>>>>
> >>>>>
> >>>>> Tempest in-tree tests
> >>>>>
> >>>>> Sean started work on it [5] and I think it's a good idea to get them
> in
> >>>>> Cinder repo to run them on Tempest jobs and 3-rd party CIs against a
> >> real
> >>>>> backend.
> >>>>>
> >>>>>
> >>>>> Functional tests for python-brick-cinderclient-ext
> >>>>>
> >>>>> There are patches that introduces functional tests [6] and new job
> [7].
> >>>>>
> >>>>>
> >>>>> Functional tests for python-cinderclient
> >>>>>
> >>>>> We've got a very limited set of such tests and non-voting job. IMO,
> we
> >>>> can
> >>>>> run them even with Cinder Fake Driver to make them not depended on a
> >>>>> storage backend and make it faster. I believe, we can make this job
> >>>> voting
> >>>>> soon. Also, we need more contributors to this kind of tests.
> >>>>>
> >>>>>
> >>>>> Integrated tests for python-cinderclient
> >>>>>
> >>>>> We need such tests to make sure that we won't break Nova, Heat or
> other
> >>>>> python-cinderclient consumers with a next merged patch. There is a
> >> thread
> >>>>> in openstack-dev ML about such tests [8] and proposal [9] to
> introduce
> >>>> them
> >>>>> to python-cinderclient.
> >>>>>
> >>>>>
> >>>>> Rally tests
> >>>>>
> >>>>> IMO, it would be good to have new Rally scenarios for every patches
> >> like
> >>>>> 'improves performance', 'fixes concurrency issues', etc. Even if we
> as
> >> a
> >>>>> Cinder community don't have enough time to implement them, we have to
> >> ask
> >>>>> for them in reviews, openstack-dev ML, file Rally bugs and blueprints
> >> if
> >>>>> needed.
> >>>>>
> >>>>
> >>>> Are there any recent examples of a fix like this recently where it
> would
> >>>> seem like a reasonable task to write a Rally scenario along with the
> >> patch?
> >>>>
> >>>> Not being very familiar with Rally (as I think most of us aren't), I'm
> >>>> having a hard time picturing this.
> >>>>
> >>>>>
> >>>>> [1] https://review.openstack.org/#/c/282861/
> >>>>> [2] http://paste.openstack.org/show/488925/
> >>>>> [3] https://review.openstack.org/#/c/267801/
> >>>>> [4] https://review.openstack.org/#/c/287115/
> >>>>> [5] https://review.openstack.org/#/c/274471/
> >>>>> [6] https://review.openstack.org/#/c/265811/
> >>>>> [7] https://review.openstack.org/#/c/265925/
> >>>>> [8]
> >>>>>
> >>>>
> >>
> http://lists.openstack.org/pipermail/openstack-dev/2016-March/088027.html
> >>>>> [9] https://review.openstack.org/#/c/279432/
> >>>>>
> >>>>>
> >>>>> Regards,
> >>>>> Ivan Kolodyazhny,
> >>>>> http://blog.e0ne.info/
> >>>>>
> >>
> >>
> >>
> __________________________________________________________________________
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >
> >
> >
> >
> __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160302/cc80120a/attachment.html>