Open Stack

Arkady,

It's not true. We've got non-voting Rally job on Cinder gates called
"gate-rally-dsvm-cinder".

Regards,
Ivan Kolodyazhny,
http://blog.e0ne.info/

On Wed, Mar 2, 2016 at 4:52 PM, <Arkady_Kanevsky at dell.com> wrote:

> Rally is not part of the gate.
>
> Also making performance test without 3rd party CI will not be very useful.
>
> It is a good idea to run Rally performance and scenario testing but
> outside gate process.
>
>
>
> *From:* Ivan Kolodyazhny [mailto:e0ne at e0ne.info]
> *Sent:* Wednesday, March 02, 2016 8:36 AM
> *To:* OpenStack Development Mailing List (not for usage questions) <
> openstack-dev at lists.openstack.org>
> *Subject:* Re: [openstack-dev] [cinder] Proposal: changes to our current
> testing process
>
>
>
> Eric,
>
>
>
> There are Gorka's patches [10] to remove API Races
>
>
>
>
>
> [10]
> https://review.openstack.org/#/q/project:openstack/cinder+branch:master+topic:fix/api-races-simplified
>
>
> Regards,
> Ivan Kolodyazhny,
> http://blog.e0ne.info/
>
>
>
> On Wed, Mar 2, 2016 at 4:27 PM, Eric Harney <eharney at redhat.com> wrote:
>
> On 03/02/2016 06:25 AM, Ivan Kolodyazhny wrote:
> > Hi Team,
> >
> > Here are my thoughts and proposals how to make Cinder testing process
> > better. I won't cover "3rd party CI's" topic here. I will share my
> opinion
> > about current and feature jobs.
> >
> >
> > Unit-tests
> >
> >    - Long-running tests. I hope, everybody will agree that unit-tests
> must
> >    be quite simple and very fast. Unit tests which takes more than 3-5
> seconds
> >    should be refactored and/or moved to 'integration' tests.
> >    Thanks to Tom Barron for several fixes like [1]. IMO, we it would be
> >    good to have some hacking checks to prevent such issues in a future.
> >
> >    - Tests coverage. We don't check it in an automatic way on gates.
>
> >    Usually, we require to add some unit-tests during code review
> process. Why
> >    can't we add coverage job to our CI and do not merge new patches, with
> >    will decrease tests coverage rate? Maybe, such job could be voting in
> a
> >    future to not ignore it. For now, there is not simple way to check
> coverage
> >    because 'tox -e cover' output is not useful [2].
> >
> >
> > Functional tests for Cinder
> >
> > We introduced some functional tests last month [3]. Here is a patch to
> > infra to add new job [4]. Because these tests were moved from
> unit-tests, I
> > think we're OK to make this job voting. Such tests should not be a
> > replacement for Tempest. They even could tests Cinder with Fake Driver to
> > make it faster and not related on storage backends issues.
> >
> >
> > Tempest in-tree tests
> >
> > Sean started work on it [5] and I think it's a good idea to get them in
> > Cinder repo to run them on Tempest jobs and 3-rd party CIs against a real
> > backend.
> >
> >
> > Functional tests for python-brick-cinderclient-ext
> >
> > There are patches that introduces functional tests [6] and new job [7].
> >
> >
> > Functional tests for python-cinderclient
> >
> > We've got a very limited set of such tests and non-voting job. IMO, we
> can
> > run them even with Cinder Fake Driver to make them not depended on a
> > storage backend and make it faster. I believe, we can make this job
> voting
> > soon. Also, we need more contributors to this kind of tests.
> >
> >
> > Integrated tests for python-cinderclient
> >
> > We need such tests to make sure that we won't break Nova, Heat or other
> > python-cinderclient consumers with a next merged patch. There is a thread
> > in openstack-dev ML about such tests [8] and proposal [9] to introduce
> them
> > to python-cinderclient.
> >
> >
> > Rally tests
> >
> > IMO, it would be good to have new Rally scenarios for every patches like
> > 'improves performance', 'fixes concurrency issues', etc. Even if we as a
> > Cinder community don't have enough time to implement them, we have to ask
> > for them in reviews, openstack-dev ML, file Rally bugs and blueprints if
> > needed.
> >
>
> Are there any recent examples of a fix like this recently where it would
> seem like a reasonable task to write a Rally scenario along with the patch?
>
> Not being very familiar with Rally (as I think most of us aren't), I'm
> having a hard time picturing this.
>
> >
> > [1] https://review.openstack.org/#/c/282861/
> > [2] http://paste.openstack.org/show/488925/
> > [3] https://review.openstack.org/#/c/267801/
> > [4] https://review.openstack.org/#/c/287115/
> > [5] https://review.openstack.org/#/c/274471/
> > [6] https://review.openstack.org/#/c/265811/
> > [7] https://review.openstack.org/#/c/265925/
> > [8]
> >
> http://lists.openstack.org/pipermail/openstack-dev/2016-March/088027.html
> > [9] https://review.openstack.org/#/c/279432/
> >
> >
> > Regards,
> > Ivan Kolodyazhny,
> > http://blog.e0ne.info/
> >
> >
> >
>
> >
> __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160302/66553af0/attachment.html>