Open Stack

On 02/03, Ivan Kolodyazhny wrote:
> I'll try to implement such scenario and step-by-step guideline soon.
> 

That would be fantastic!! Thank you very much

Looking forward to it.  :-)

Cheers,
Gorka.

> Regards,
> Ivan Kolodyazhny,
> http://blog.e0ne.info/
> 
> On Wed, Mar 2, 2016 at 5:16 PM, Eric Harney <eharney at redhat.com> wrote:
> 
> > On 03/02/2016 10:07 AM, Ivan Kolodyazhny wrote:
> > > Eric,
> > >
> > > For now, we test Cinder API with some concurrency only with Rally, so,
> > IMO,
> > > it's reasonable get more scenarios for API races fixes.
> > >
> > > It's not a hard task to implement new scenarios, they are pretty simple:
> > > [11] and [12]
> > >
> >
> > Sure, these are simple, but I think it's nowhere near that simple to
> > write a scenario which will prove that "remove API races" works correctly.
> >
> > > [11]
> > >
> > https://github.com/openstack/rally/blob/master/rally/plugins/openstack/scenarios/cinder/volumes.py#L535
> > > [12]
> > >
> > https://github.com/openstack/rally/blob/master/rally-jobs/cinder.yaml#L516
> > >
> > > Regards,
> > > Ivan Kolodyazhny,
> > > http://blog.e0ne.info/
> > >
> > > On Wed, Mar 2, 2016 at 4:50 PM, Eric Harney <eharney at redhat.com> wrote:
> > >
> > >> On 03/02/2016 09:36 AM, Ivan Kolodyazhny wrote:
> > >>> Eric,
> > >>>
> > >>> There are Gorka's patches [10] to remove API Races
> > >>>
> > >>>
> > >>> [10]
> > >>>
> > >>
> > https://review.openstack.org/#/q/project:openstack/cinder+branch:master+topic:fix/api-races-simplified
> > >>>
> > >>> Regards,
> > >>> Ivan Kolodyazhny,
> > >>> http://blog.e0ne.info/
> > >>>
> > >>
> > >> So the second part of my question is, is writing a Rally job to prove
> > >> out that code a reasonable task?
> > >>
> > >> How hard is that to do and what does it look like?
> > >>
> > >>> On Wed, Mar 2, 2016 at 4:27 PM, Eric Harney <eharney at redhat.com>
> > wrote:
> > >>>
> > >>>> On 03/02/2016 06:25 AM, Ivan Kolodyazhny wrote:
> > >>>>> Hi Team,
> > >>>>>
> > >>>>> Here are my thoughts and proposals how to make Cinder testing process
> > >>>>> better. I won't cover "3rd party CI's" topic here. I will share my
> > >>>> opinion
> > >>>>> about current and feature jobs.
> > >>>>>
> > >>>>>
> > >>>>> Unit-tests
> > >>>>>
> > >>>>>    - Long-running tests. I hope, everybody will agree that unit-tests
> > >>>> must
> > >>>>>    be quite simple and very fast. Unit tests which takes more than
> > 3-5
> > >>>> seconds
> > >>>>>    should be refactored and/or moved to 'integration' tests.
> > >>>>>    Thanks to Tom Barron for several fixes like [1]. IMO, we it would
> > be
> > >>>>>    good to have some hacking checks to prevent such issues in a
> > future.
> > >>>>>
> > >>>>>    - Tests coverage. We don't check it in an automatic way on gates.
> > >>>>>    Usually, we require to add some unit-tests during code review
> > >>>> process. Why
> > >>>>>    can't we add coverage job to our CI and do not merge new patches,
> > >> with
> > >>>>>    will decrease tests coverage rate? Maybe, such job could be voting
> > >> in
> > >>>> a
> > >>>>>    future to not ignore it. For now, there is not simple way to check
> > >>>> coverage
> > >>>>>    because 'tox -e cover' output is not useful [2].
> > >>>>>
> > >>>>>
> > >>>>> Functional tests for Cinder
> > >>>>>
> > >>>>> We introduced some functional tests last month [3]. Here is a patch
> > to
> > >>>>> infra to add new job [4]. Because these tests were moved from
> > >>>> unit-tests, I
> > >>>>> think we're OK to make this job voting. Such tests should not be a
> > >>>>> replacement for Tempest. They even could tests Cinder with Fake
> > Driver
> > >> to
> > >>>>> make it faster and not related on storage backends issues.
> > >>>>>
> > >>>>>
> > >>>>> Tempest in-tree tests
> > >>>>>
> > >>>>> Sean started work on it [5] and I think it's a good idea to get them
> > in
> > >>>>> Cinder repo to run them on Tempest jobs and 3-rd party CIs against a
> > >> real
> > >>>>> backend.
> > >>>>>
> > >>>>>
> > >>>>> Functional tests for python-brick-cinderclient-ext
> > >>>>>
> > >>>>> There are patches that introduces functional tests [6] and new job
> > [7].
> > >>>>>
> > >>>>>
> > >>>>> Functional tests for python-cinderclient
> > >>>>>
> > >>>>> We've got a very limited set of such tests and non-voting job. IMO,
> > we
> > >>>> can
> > >>>>> run them even with Cinder Fake Driver to make them not depended on a
> > >>>>> storage backend and make it faster. I believe, we can make this job
> > >>>> voting
> > >>>>> soon. Also, we need more contributors to this kind of tests.
> > >>>>>
> > >>>>>
> > >>>>> Integrated tests for python-cinderclient
> > >>>>>
> > >>>>> We need such tests to make sure that we won't break Nova, Heat or
> > other
> > >>>>> python-cinderclient consumers with a next merged patch. There is a
> > >> thread
> > >>>>> in openstack-dev ML about such tests [8] and proposal [9] to
> > introduce
> > >>>> them
> > >>>>> to python-cinderclient.
> > >>>>>
> > >>>>>
> > >>>>> Rally tests
> > >>>>>
> > >>>>> IMO, it would be good to have new Rally scenarios for every patches
> > >> like
> > >>>>> 'improves performance', 'fixes concurrency issues', etc. Even if we
> > as
> > >> a
> > >>>>> Cinder community don't have enough time to implement them, we have to
> > >> ask
> > >>>>> for them in reviews, openstack-dev ML, file Rally bugs and blueprints
> > >> if
> > >>>>> needed.
> > >>>>>
> > >>>>
> > >>>> Are there any recent examples of a fix like this recently where it
> > would
> > >>>> seem like a reasonable task to write a Rally scenario along with the
> > >> patch?
> > >>>>
> > >>>> Not being very familiar with Rally (as I think most of us aren't), I'm
> > >>>> having a hard time picturing this.
> > >>>>
> > >>>>>
> > >>>>> [1] https://review.openstack.org/#/c/282861/
> > >>>>> [2] http://paste.openstack.org/show/488925/
> > >>>>> [3] https://review.openstack.org/#/c/267801/
> > >>>>> [4] https://review.openstack.org/#/c/287115/
> > >>>>> [5] https://review.openstack.org/#/c/274471/
> > >>>>> [6] https://review.openstack.org/#/c/265811/
> > >>>>> [7] https://review.openstack.org/#/c/265925/
> > >>>>> [8]
> > >>>>>
> > >>>>
> > >>
> > http://lists.openstack.org/pipermail/openstack-dev/2016-March/088027.html
> > >>>>> [9] https://review.openstack.org/#/c/279432/
> > >>>>>
> > >>>>>
> > >>>>> Regards,
> > >>>>> Ivan Kolodyazhny,
> > >>>>> http://blog.e0ne.info/
> > >>>>>
> > >>
> > >>
> > >>
> > __________________________________________________________________________
> > >> OpenStack Development Mailing List (not for usage questions)
> > >> Unsubscribe:
> > OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > >>
> > >
> > >
> > >
> > >
> > __________________________________________________________________________
> > > OpenStack Development Mailing List (not for usage questions)
> > > Unsubscribe:
> > OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > >
> >
> >
> > __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >

> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev