<div dir="ltr">Hi all, <div><br></div><div>This thread is very useful. We've detect issue related to the mission statement and name of proposed program on early steps. Seems like mission statement and name are totally unclear and don't present in the right perspective goals of this program. </div>
<div><br></div><div>I updated name and mission statement:</div><div><br></div><div>name:</div><div> SLA Management</div><div><br></div><div>mission:</div><div><div> Provide SLA Management for production OpenStack clouds. This includes</div>
<div> measuring and tracking performance of OpenStack Services, key API methods</div><div> and cloud applications, performance and functional tests on demand, and</div><div> everything that is required to detect and debug issues in live production clouds. </div>
</div><div><br></div><div><div>As well, I updated patch to governance:</div><div><a href="https://review.openstack.org/#/c/108502/3">https://review.openstack.org/#/c/108502/3</a><br></div></div><div><br></div><div><br></div>
<div>I hope now it's more clear, what is the goal of this program and why we should add new program. </div><div><br></div><div>Thoughts? </div><div><br></div><div>Best regards,</div><div>Boris Pavlovic </div></div><div class="gmail_extra">
<br><br><div class="gmail_quote">On Tue, Jul 29, 2014 at 12:39 AM, Boris Pavlovic <span dir="ltr"><<a href="mailto:boris@pavlovic.me" target="_blank">boris@pavlovic.me</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr"><div>Hi Sean,</div><div><br></div><div>I appreciate you valuing Rally so highly as to suggesting it should join the QA program. It is a great vote of confidence for me. While I believe that Rally and Tempest will always work closely together, the intended utility and the direction of where we are planing to take Rally will not be compatible with the direction of where I think the QA program is going. Please let me explain in more details below.</div>
<div><br></div><div>Tempest is a collection of Functional and Performance Tests which is used by the developers to improve the quality of the OpenStack code. </div><div><br></div><div>Rally on the other hand, is envisioned as a Tool that is going to be run by the cloud operators in order to measure, tune and continuously improve the performance of an OpenStack cloud. Moreover, we have an SLA module that allows the Operator to define what constitutes an acceptable level of performance and a profiler that would provide both the user and the developer the diagnostic set of performance data. Finally, Rally is designed to run on production clouds and to be integrated as a Horizon plugin. </div>
<div><br></div><div>In the future, we envision integrating Rally with other services (e.g. Logging as a Service, Satori, Rubick, and other operator-targeted services). I believe that this is not the direction compatible with the mission of the the QA program .</div>
<div><br></div><div>Before applying for a new Performance and Scalability program, we have thought that the best existing program that Rally could be a part of now and in the future is the Telemetry program. We have discussed with Eoghan Glynn the idea of extending the scope of its mission to include other operator related projects and include Rally to it. Eoghan liked the idea in general but felt that Ceilometer currently has too much on its plate and was not in a position to merge in a new project. However, I can still see the two programs maturing and potentially becoming one down the road. </div>
<div><br></div><div>Now, regarding the point that you make of Rally and Tempest doing some duplicate work. I completely agree with you that we should avoid it as much as possible and we should stay in close communication to make sure that duplicate requirements are only implemented once. </div>
<div><br></div><div>Following our earlier discussion, Rally is now using Tempest for those benchmarks that do not require special complex environments, we also encapsulated and automated Tempest usage to make it more accessible for the Operators (here is the Blog documenting it -- <a href="http://www.mirantis.com/blog/rally-openstack-tempest-testing-made-simpler/" target="_blank">http://www.mirantis.com/blog/rally-openstack-tempest-testing-made-simpler/</a>). </div>
<div><br></div><div>We would like to further continue to de-duplicate the work inside Tempest and Rally. We made some joint design decisions in Atlanta to transfer some of the Integration code from Rally to Tempest, resulting in the work performed by Andrew Kurilin (<a href="https://review.openstack.org/#/c/94473/" target="_blank">https://review.openstack.org/#/c/94473/</a>). I would encourage and welcome more of such cooperation in the future. </div>
<div><br></div><div>I trust that this addresses most of your concerns and please do not hesitate to bring up more questions and suggestions.</div><div><br></div><div>Sincerely,</div><div><br></div><div>Boris</div></div><div class="gmail_extra">
<br><br><div class="gmail_quote"><div><div class="h5">On Sun, Jul 27, 2014 at 6:57 PM, Sean Dague <span dir="ltr"><<a href="mailto:sean@dague.net" target="_blank">sean@dague.net</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><div class="h5">
<div><div>On 07/26/2014 05:51 PM, Hayes, Graham wrote:<br>
> On Tue, 2014-07-22 at 12:18 -0400, Sean Dague wrote:<br>
>> On 07/22/2014 11:58 AM, David Kranz wrote:<br>
>>> On 07/22/2014 10:44 AM, Sean Dague wrote:<br>
>>>> Honestly, I'm really not sure I see this as a different program, but is<br>
>>>> really something that should be folded into the QA program. I feel like<br>
>>>> a top level effort like this is going to lead to a lot of duplication in<br>
>>>> the data analysis that's currently going on, as well as functionality<br>
>>>> for better load driver UX.<br>
>>>><br>
>>>> -Sean<br>
>>> +1<br>
>>> It will also lead to pointless discussions/arguments about which<br>
>>> activities are part of "QA" and which are part of<br>
>>> "Performance and Scalability Testing".<br>
><br>
> I think that those discussions will still take place, it will just be on<br>
> a per repository basis, instead of a per program one.<br>
><br>
> [snip]<br>
><br>
>><br>
>> Right, 100% agreed. Rally would remain with it's own repo + review team,<br>
>> just like grenade.<br>
>><br>
>> -Sean<br>
>><br>
><br>
> Is the concept of a separate review team not the point of a program?<br>
><br>
> In the the thread from Designate's Incubation request Thierry said [1]:<br>
><br>
>> "Programs" just let us bless goals and teams and let them organize<br>
>> code however they want, with contribution to any code repo under that<br>
>> umbrella being considered "official" and ATC-status-granting.<br>
><br>
> I do think that this is something that needs to be clarified by the TC -<br>
> Rally could not get a PTL if they were part of the QA project, but every<br>
> time we get a program request, the same discussion happens.<br>
><br>
> I think that mission statements can be edited to fit new programs as<br>
> they occur, and that it is more important to let teams that have been<br>
> working closely together to stay as a distinct group.<br>
<br>
</div></div>My big concern here is that many of the things that these efforts have<br>
been doing are things we actually want much closer to the base. For<br>
instance, metrics on Tempest runs.<br>
<br>
When Rally was first created it had it's own load generator. It took a<br>
ton of effort to keep the team from duplicating that and instead just<br>
use some subset of Tempest. Then when measuring showed up, we actually<br>
said that is something that would be great in Tempest, so whoever ran<br>
it, be it for Testing, Monitoring, or Performance gathering, would have<br>
access to that data. But the Rally team went off in a corner and did it<br>
otherwise. That's caused the QA team to have to go and redo this work<br>
from scratch with subunit2sql, in a way that can be consumed by multiple<br>
efforts.<br>
<br>
So I'm generally -1 to this being a separate effort on the basis that so<br>
far the team has decided to stay in their own sandbox instead of<br>
participating actively where many of us thing the functions should be<br>
added. I also think this isn't like Designate, because this isn't<br>
intended to be part of the integrated release.<br>
<br>
Of course you could decide to slice up the universe in a completely<br>
different way, but we have toolchains today, which I think the focus<br>
should be on participating there.<br>
<div><div><br>
-Sean<br>
<br>
--<br>
Sean Dague<br>
<a href="http://dague.net" target="_blank">http://dague.net</a><br>
<br>
</div></div><br></div></div><div class="">_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></div></blockquote></div><br></div>
</blockquote></div><br></div>