Open Stack

----- Original Message -----
From: "Thierry Carrez" <thierry at openstack.org>
To: "OpenStack Development Mailing List" <openstack-dev at lists.openstack.org>
Sent: Wednesday, August 6, 2014 4:00:35 PM
Subject: [openstack-dev] Which program for Rally


1. Rally as an essential QA tool
Performance testing (and especially performance regression testing) is
an essential QA function, and a feature that Rally provides. If the QA
team is happy to use Rally to fill that function, then Rally can
obviously be adopted by the (already-existing) QA program. That said,
that would put Rally under the authority of the QA PTL, and that raises
a few questions due to the current architecture of Rally, which is more
product-oriented. There needs to be further discussion between the QA
core team and the Rally team to see how that could work and if that
option would be acceptable for both sides.


I want to share an use case of Rally for performance bench-marking. 
I use Rally to benchmark Keystone performance. I can easily get results
for comparison between different configurations, openstack distributions
etc. Here is a sample result :-

https://github.com/stackforge/rally/blob/master/doc/user_stories/keystone/authenticate.rst

IMO Rally can play a essential role in performance regression testing. 


Regards,
Neependra Khare
Performance Engineering @ Red Hat


2. Rally as an essential operator tool
Regular benchmarking of OpenStack deployments is a best practice for
cloud operators, and a feature that Rally provides. With a bit of a
stretch, we could consider that benchmarking is essential to the
completion of the OpenStack project mission. That program could one day
evolve to include more such "operations best practices" tools. In
addition to the slight stretch already mentioned, one concern here is
that we still want to have performance testing in QA (which is clearly
essential to the production of "OpenStack"). Letting Rally primarily be
an operational tool might make that outcome more difficult.

3. Let Rally be a product on top of OpenStack
The last option is to not have Rally in any program, and not consider it
*essential* to the production of the "OpenStack" integrated release or
the completion of the OpenStack project mission. Rally can happily exist
as an operator tool on top of OpenStack. It is built as a monolithic
product: that approach works very well for external complementary
solutions... Also be more integrated in OpenStack or part of the
OpenStack programs might come at a cost (slicing some functionality out
of rally to make it more a framework and less a product) that might not
be what its authors want.

Let's explore each option to see which ones are viable, and the pros and
cons of each.

-- 
Thierry Carrez (ttx)

_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev