[openstack-dev] Which program for Rally

Anne Gentle anne at openstack.org
Fri Aug 8 14:41:00 UTC 2014

On Wed, Aug 6, 2014 at 5:30 AM, Thierry Carrez <thierry at openstack.org>

> Hi everyone,
> At the TC meeting yesterday we discussed Rally program request and
> incubation request. We quickly dismissed the incubation request, as
> Rally appears to be able to live happily on top of OpenStack and would
> benefit from having a release cycle decoupled from the OpenStack
> "integrated release".
> That leaves the question of the program. OpenStack programs are created
> by the Technical Committee, to bless existing efforts and teams that are
> considered *essential* to the production of the "OpenStack" integrated
> release and the completion of the OpenStack project mission. There are 3
> ways to look at Rally and official programs at this point:
> 1. Rally as an essential QA tool
> Performance testing (and especially performance regression testing) is
> an essential QA function, and a feature that Rally provides. If the QA
> team is happy to use Rally to fill that function, then Rally can
> obviously be adopted by the (already-existing) QA program. That said,
> that would put Rally under the authority of the QA PTL, and that raises
> a few questions due to the current architecture of Rally, which is more
> product-oriented. There needs to be further discussion between the QA
> core team and the Rally team to see how that could work and if that
> option would be acceptable for both sides.

Pros: Performance testing is great and we don't have it now that I know of.
- QA then takes on more scope in their mission. Do they want it?
- Is Rally actually splittable this way?
- How important is the PTL role - PTL cage match may ensue next election?

> 2. Rally as an essential operator tool
> Regular benchmarking of OpenStack deployments is a best practice for
> cloud operators, and a feature that Rally provides. With a bit of a
> stretch, we could consider that benchmarking is essential to the
> completion of the OpenStack project mission. That program could one day
> evolve to include more such "operations best practices" tools. In
> addition to the slight stretch already mentioned, one concern here is
> that we still want to have performance testing in QA (which is clearly
> essential to the production of "OpenStack"). Letting Rally primarily be
> an operational tool might make that outcome more difficult.

Pros: Great start to an operator program for tooling.

- Would have to ensure Rally is what we want "first" as getting to be PTL
since you are first to propose seems to be the model.
- Is benchmark testing and SLA-meeting a best first tool? Or monitoring? Or
deployment? Or some other tools?
- Is this program what operators want?

> 3. Let Rally be a product on top of OpenStack
> The last option is to not have Rally in any program, and not consider it
> *essential* to the production of the "OpenStack" integrated release or
> the completion of the OpenStack project mission. Rally can happily exist
> as an operator tool on top of OpenStack. It is built as a monolithic
> product: that approach works very well for external complementary
> solutions... Also be more integrated in OpenStack or part of the
> OpenStack programs might come at a cost (slicing some functionality out
> of rally to make it more a framework and less a product) that might not
> be what its authors want.
Pros: Rally can set the standards for this path and lead on this pioneer

- Would this tool be applied against continuously-deployed clouds?
- Is there any preference or advantage to be outside of integrated releases?
- Will people believe it's official?

Hopefully that summarizes how I'm looking at this application --

> Let's explore each option to see which ones are viable, and the pros and
> cons of each.
> --
> Thierry Carrez (ttx)
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140808/4eaeb644/attachment.html>

More information about the OpenStack-dev mailing list