Open Stack

On Fri, Aug 15, 2014 at 01:57:29AM +0400, Boris Pavlovic wrote:
> Matt,
> 
>> One thing did just occur to me while writing this though it's probably worth
> > investigating splitting out the stress test framework as an external
> > tool/project after we start work on the tempest library. [3]
> 
> 
> 
> I fully agree with the fact that stress testing doesn't belong to Tempest.
> 
> This current thread is all about this aspect and all my arguments, related
> to splitting Rally and merging it to tempest are related to this.
> 
> 
> Could you please elaborate, why instead of ready solution "Rally"  that has
> community and is aligned with OpenStack and OpenStack processes you are
> going to create from scratch similar solution?

This is the same issue which was brought up on your subunit2sql thread [1] and
has come up several times already in this thread. Rally doesn't work as
individual components, you need to use the whole tool chain to do anything with
it. There are pieces of Rally which would be very useful in conjunction with all
the other projects we have in the gating workflow. But, because Rally has
decided to duplicate much of what we already have and then tie everything
together in a monolithic toolchain we can't use these pieces which are useful by
itself. This doesn't work with the workflow we already have, it also ignores the
split in functionality we currently have and are working on improving. Instead
of rewriting all the details again just look at [2][3] they elaborate on all of
this already.

You're also ignoring that by splitting out the stress test framework we'd be
doing the exact thing you're so opposed to doing in rally. Which is splitting
out existing functionality from a larger more complex project into smaller more
purpose built consumable chunks that work together to build a modular pipeline,
which can be used in different configurations to suit the application. It's also
not being created from scratch, the stress framework already exists and has for
quite some time. (it pre-dates Rally) We would just be breaking it off into a
separate repo to make the split in functionality more clear. It is essentially
a separate tool already, it just lives in the tempest tree which I feel is a
source of some confusion around it.

> 
> I really don't see any reasons why we need to duplicate existing and
> working solution and can't just work together on Rally?

So I have to point out the double standard with this statement. You're ignoring
all the functionality that Rally has already duplicated. For example, the fact
that tempest exists as a battery of tests which is being slowly duplicated in
Rally, or the stress test framework, which is functionality that was more or
less completely duplicated in Rally. You seem to be under the mistaken
impression that by continuing to improve things on QA program projects we're
duplicating Rally. Which is honestly something I'm having a hard time
understanding. I especially do not see the value in working on improvements to
tempest and the existing workflow by replacing everything with Rally.

--------

So I think there's been continued confusion around exactly how all the projects
work together in the QA program. So I wrote a blog post giving a high level
overview here: http://blog.kortar.org/?p=4

Now if Rally is willing to work with us it would really be awesome since a lot
of the work has already been done by them. But we would need to rework Rally so
the bits which don't already exist in the QA program are interoperable with what
we have now and can be used independently. Duplicated functionality would need
to be removed, and we would need to add the improvements in Rally on existing
functionality back into the projects, where they really belong. However, as I've
been continually saying in this thread Rally in it's current form doesn't work
with our current model, nor does it work with the vision, at least I have, for
how things should be in the future.

Now I'm sure someone is going to see the flow diagrams on my blog and see it as
a challenge to explain how Rally does it better today, or something along those
lines. Please don't, because honestly it's irrelevant, these are just ideas in
my head, and where I want to help steer the QA program as long as I'm working on
it. (at least as of today) I fully expect things to evolve and grow more
organically resulting in something that will be completely different from that.

Also, I'm really done with this thread, I've outlined my stance repeatedly and
tried my best to explain my position as clearly as I can. At this point I have
nothing else to add. I view the burden as being fully on the Rally team to
decide whether they want to start working with the QA program towards
integrating Rally into the QA program, (the steps Sean outlined in [2] are a
good start) or remain a separate external project. (or I guess unless the TC
mandates something else)

One last comment, I do want to apologize in advance if any of my words seem
harsh or antagonistic, that's really not my intent.

-Matt Treinish

[1] http://lists.openstack.org/pipermail/openstack-dev/2014-August/041815.html
[2] http://lists.openstack.org/pipermail/openstack-dev/2014-August/042069.html
[3] http://lists.openstack.org/pipermail/openstack-dev/2014-August/042269.html

<snip>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140818/e2e1823c/attachment.pgp>