<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Le 30/07/2014 22:13, Boris Pavlovic a
écrit :<br>
</div>
<blockquote
cite="mid:CAD85om3j2K1Hy=TuFuimnP+zK5Hh9mOQR6oZ-rKqMNpYLc74dw@mail.gmail.com"
type="cite">
<div dir="ltr">Hi all,
<div><br>
</div>
<div>This thread is very useful. We've detect issue related to
the mission statement and name of proposed program on early
steps. Seems like mission statement and name are totally
unclear and don't present in the right perspective goals of
this program. </div>
<div><br>
</div>
<div>I updated name and mission statement:</div>
<div><br>
</div>
<div>name:</div>
<div> SLA Management</div>
<div><br>
</div>
<div>mission:</div>
<div>
<div> Provide SLA Management for production OpenStack
clouds. This includes</div>
<div> measuring and tracking performance of OpenStack
Services, key API methods</div>
<div> and cloud applications, performance and functional
tests on demand, and</div>
<div> everything that is required to detect and debug
issues in live production clouds. </div>
</div>
<div><br>
</div>
<div>
<div>As well, I updated patch to governance:</div>
<div><a moz-do-not-send="true"
href="https://review.openstack.org/#/c/108502/3">https://review.openstack.org/#/c/108502/3</a><br>
</div>
</div>
<div><br>
</div>
<div><br>
</div>
<div>I hope now it's more clear, what is the goal of this
program and why we should add new program. </div>
<div><br>
</div>
<div>Thoughts? </div>
<div><br>
</div>
</div>
</blockquote>
<br>
<br>
-1 to it. SLA means that you create a contract in between a provider
and an user. Here, you don't create a contract (ie. you don't create
ratios and engagement) but you monitor these contracts.<br>
<br>
As there are no SLAs now in OpenStack upstream (that's something for
operators), we can't say if the KPIs are OK or not. <br>
How can you ensure that the code will be 9 nines if you don't look
at how OpenStack will be deployed ?<br>
<br>
If you say the mission statement is to provide measurement tools for
OpenStack, then it possibly goes either in QA or in Telemetry
programs. If you say that the goal is to detect and debug issues in
clouds, then it clearly goes into QA program.<br>
<br>
<br>
IMHO, both your name and your mission statement are confusing. Long
story short, I'm pro Rally as a separate project but in the QA
program. <br>
<br>
-Sylvain<br>
<br>
<blockquote
cite="mid:CAD85om3j2K1Hy=TuFuimnP+zK5Hh9mOQR6oZ-rKqMNpYLc74dw@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>Best regards,</div>
<div>Boris Pavlovic </div>
</div>
<div class="gmail_extra">
<br>
<br>
<div class="gmail_quote">On Tue, Jul 29, 2014 at 12:39 AM, Boris
Pavlovic <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:boris@pavlovic.me" target="_blank">boris@pavlovic.me</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div>Hi Sean,</div>
<div><br>
</div>
<div>I appreciate you valuing Rally so highly as to
suggesting it should join the QA program. It is a great
vote of confidence for me. While I believe that Rally
and Tempest will always work closely together, the
intended utility and the direction of where we are
planing to take Rally will not be compatible with the
direction of where I think the QA program is going.
Please let me explain in more details below.</div>
<div><br>
</div>
<div>Tempest is a collection of Functional and Performance
Tests which is used by the developers to improve the
quality of the OpenStack code. </div>
<div><br>
</div>
<div>Rally on the other hand, is envisioned as a Tool that
is going to be run by the cloud operators in order to
measure, tune and continuously improve the performance
of an OpenStack cloud. Moreover, we have an SLA module
that allows the Operator to define what constitutes an
acceptable level of performance and a profiler that
would provide both the user and the developer the
diagnostic set of performance data. Finally, Rally is
designed to run on production clouds and to be
integrated as a Horizon plugin. </div>
<div><br>
</div>
<div>In the future, we envision integrating Rally with
other services (e.g. Logging as a Service, Satori,
Rubick, and other operator-targeted services). I believe
that this is not the direction compatible with the
mission of the the QA program .</div>
<div><br>
</div>
<div>Before applying for a new Performance and Scalability
program, we have thought that the best existing program
that Rally could be a part of now and in the future is
the Telemetry program. We have discussed with Eoghan
Glynn the idea of extending the scope of its mission to
include other operator related projects and include
Rally to it. Eoghan liked the idea in general but felt
that Ceilometer currently has too much on its plate and
was not in a position to merge in a new project.
However, I can still see the two programs maturing and
potentially becoming one down the road. </div>
<div><br>
</div>
<div>Now, regarding the point that you make of Rally and
Tempest doing some duplicate work. I completely agree
with you that we should avoid it as much as possible and
we should stay in close communication to make sure that
duplicate requirements are only implemented once. </div>
<div><br>
</div>
<div>Following our earlier discussion, Rally is now using
Tempest for those benchmarks that do not require special
complex environments, we also encapsulated and automated
Tempest usage to make it more accessible for the
Operators (here is the Blog documenting it -- <a
moz-do-not-send="true"
href="http://www.mirantis.com/blog/rally-openstack-tempest-testing-made-simpler/"
target="_blank">http://www.mirantis.com/blog/rally-openstack-tempest-testing-made-simpler/</a>). </div>
<div><br>
</div>
<div>We would like to further continue to de-duplicate the
work inside Tempest and Rally. We made some joint design
decisions in Atlanta to transfer some of the Integration
code from Rally to Tempest, resulting in the work
performed by Andrew Kurilin (<a moz-do-not-send="true"
href="https://review.openstack.org/#/c/94473/"
target="_blank">https://review.openstack.org/#/c/94473/</a>).
I would encourage and welcome more of such cooperation
in the future. </div>
<div><br>
</div>
<div>I trust that this addresses most of your concerns and
please do not hesitate to bring up more questions and
suggestions.</div>
<div><br>
</div>
<div>Sincerely,</div>
<div><br>
</div>
<div>Boris</div>
</div>
<div class="gmail_extra">
<br>
<br>
<div class="gmail_quote">
<div>
<div class="h5">On Sun, Jul 27, 2014 at 6:57 PM, Sean
Dague <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:sean@dague.net" target="_blank">sean@dague.net</a>></span>
wrote:<br>
</div>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div class="h5">
<div>
<div>On 07/26/2014 05:51 PM, Hayes, Graham
wrote:<br>
> On Tue, 2014-07-22 at 12:18 -0400, Sean
Dague wrote:<br>
>> On 07/22/2014 11:58 AM, David Kranz
wrote:<br>
>>> On 07/22/2014 10:44 AM, Sean
Dague wrote:<br>
>>>> Honestly, I'm really not sure
I see this as a different program, but is<br>
>>>> really something that should
be folded into the QA program. I feel like<br>
>>>> a top level effort like this
is going to lead to a lot of duplication in<br>
>>>> the data analysis that's
currently going on, as well as functionality<br>
>>>> for better load driver UX.<br>
>>>><br>
>>>> -Sean<br>
>>> +1<br>
>>> It will also lead to pointless
discussions/arguments about which<br>
>>> activities are part of "QA" and
which are part of<br>
>>> "Performance and Scalability
Testing".<br>
><br>
> I think that those discussions will still
take place, it will just be on<br>
> a per repository basis, instead of a per
program one.<br>
><br>
> [snip]<br>
><br>
>><br>
>> Right, 100% agreed. Rally would
remain with it's own repo + review team,<br>
>> just like grenade.<br>
>><br>
>> -Sean<br>
>><br>
><br>
> Is the concept of a separate review team
not the point of a program?<br>
><br>
> In the the thread from Designate's
Incubation request Thierry said [1]:<br>
><br>
>> "Programs" just let us bless goals
and teams and let them organize<br>
>> code however they want, with
contribution to any code repo under that<br>
>> umbrella being considered "official"
and ATC-status-granting.<br>
><br>
> I do think that this is something that
needs to be clarified by the TC -<br>
> Rally could not get a PTL if they were
part of the QA project, but every<br>
> time we get a program request, the same
discussion happens.<br>
><br>
> I think that mission statements can be
edited to fit new programs as<br>
> they occur, and that it is more important
to let teams that have been<br>
> working closely together to stay as a
distinct group.<br>
<br>
</div>
</div>
My big concern here is that many of the things
that these efforts have<br>
been doing are things we actually want much closer
to the base. For<br>
instance, metrics on Tempest runs.<br>
<br>
When Rally was first created it had it's own load
generator. It took a<br>
ton of effort to keep the team from duplicating
that and instead just<br>
use some subset of Tempest. Then when measuring
showed up, we actually<br>
said that is something that would be great in
Tempest, so whoever ran<br>
it, be it for Testing, Monitoring, or Performance
gathering, would have<br>
access to that data. But the Rally team went off
in a corner and did it<br>
otherwise. That's caused the QA team to have to go
and redo this work<br>
from scratch with subunit2sql, in a way that can
be consumed by multiple<br>
efforts.<br>
<br>
So I'm generally -1 to this being a separate
effort on the basis that so<br>
far the team has decided to stay in their own
sandbox instead of<br>
participating actively where many of us thing the
functions should be<br>
added. I also think this isn't like Designate,
because this isn't<br>
intended to be part of the integrated release.<br>
<br>
Of course you could decide to slice up the
universe in a completely<br>
different way, but we have toolchains today, which
I think the focus<br>
should be on participating there.<br>
<div>
<div><br>
-Sean<br>
<br>
--<br>
Sean Dague<br>
<a moz-do-not-send="true"
href="http://dague.net" target="_blank">http://dague.net</a><br>
<br>
</div>
</div>
<br>
</div>
</div>
<div class="">_______________________________________________<br>
OpenStack-dev mailing list<br>
<a moz-do-not-send="true"
href="mailto:OpenStack-dev@lists.openstack.org"
target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a moz-do-not-send="true"
href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev"
target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
OpenStack-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
</blockquote>
<br>
</body>
</html>