Open Stack

Hi Angus,

Thanks for doing the tests with convergence. We are now assured that
convergence has not impacted the performance in a negative way. Given
that, in convergence, a stack provisioning process goes through a lot of
RPC calls, it puts a lot of load on the message broker and the request
looses time in network traversal etc., and in effect would hamper the
performance. As the results show, having more than 2 engines will always
yield better results with convergence. Since the deployments usually
have 2 or more engines, this works in favor of convergence.

I have always held that convergence is more for scale (how much/many)
than for performance (response time), due to it's design of distributing
load (resource provisioning from single stack) among heat engines and
also due to the fact that heat actually spends a lot of time waiting for
the delegated resource request to be completed, not doing much
computation. However, with these tests, we can eliminate any
apprehension of performance issues which would have inadvertently
sneaked in, with our focus more on scalability and reliability, than on
performance.

I was thinking we should be doing some scale testing where we have many
bigger stacks provisioned and compare the results with legacy, where we
measure memory, CPU and network bandwidth.

--
Anant


On Fri, Aug 28, 2015 at 2:45 PM, Angus Salkeld <asalkeld at mirantis.com>
wrote:

> On Fri, Aug 28, 2015 at 6:35 PM Sergey Lukjanov <slukjanov at mirantis.com>
> wrote:
>
>> Hi,
>>
>> great, it seems like migration to convergence could happen soon.
>>
>> How many times you were running each test case? Does time changing with
>> number of iterations? Are you planning to test parallel stacks creation?
>>
>
> Given the test matrix convergence/non-convergence and 2,4,8 engines, I
> have not done a lot of iterations - it's just time consuming. I might kill
> off the 2-engine case to gain more iterations.
> But from what I have observed the duration does not vary significantly.
>
> I'll test smaller stacks with lots of iterations and with a high
> concurrency. All this testing is currently on just one host so it is
> somewhat limited. Hopefully this is at least giving a useful comparison
> with these limitations.
>
> -Angus
>
>
>>
>> Thanks.
>>
>> On Fri, Aug 28, 2015 at 10:17 AM, Sergey Kraynev <skraynev at mirantis.com>
>> wrote:
>>
>>> Angus!
>>>
>>> it's Awesome!  Thank you for the investigation.
>>> I had a talk with guys from Sahara team and we decided to start testing
>>> convergence with Sahara after L release.
>>> I suppose, that Murano can also join to this process.
>>>
>>> Also AFAIK Sahara team plan to create functional tests with Heat-engine.
>>> We may add it as a non-voting job for our gate.
>>> Probably it will be good to have two different type of this job: with
>>> convergence and with default Heat.
>>>
>>> On 28 August 2015 at 04:35, Angus Salkeld <asalkeld at mirantis.com> wrote:
>>>
>>>> Hi
>>>>
>>>> I have been running some rally tests against convergence and our
>>>> existing implementation to compare.
>>>>
>>>> So far I have done the following:
>>>>
>>>>    1. defined a template with a resource group
>>>>    https://github.com/asalkeld/convergence-rally/blob/master/templates/resource_group_test_resource.yaml.template
>>>>    2. the inner resource looks like this:
>>>>    https://github.com/asalkeld/convergence-rally/blob/master/templates/server_with_volume.yaml.template (it
>>>>    uses TestResource to attempt to be a reasonable simulation of a
>>>>    server+volume+floatingip)
>>>>    3. defined a rally job:
>>>>    https://github.com/asalkeld/convergence-rally/blob/master/increasing_resources.yaml that
>>>>    creates X resources then updates to X*2 then deletes.
>>>>    4. I then ran the above with/without convergence and with 2,4,8
>>>>    heat-engines
>>>>
>>>> Here are the results compared:
>>>>
>>>> https://docs.google.com/spreadsheets/d/12kRtPsmZBl_y78aw684PTBg3op1ftUYsAEqXBtT800A/edit?usp=sharing
>>>>
>>>
>>> Results look pretty nice (especially for create) :)
>>> The strange thing for me: why on update "8 engines" shows worse results
>>> then with "4 engines"? (may be mistake in graph... ?)
>>>
>>>
>>>
>>>>
>>>>
>>>> Some notes on the results so far:
>>>>
>>>>    -  convergence with only 2 engines does suffer from RPC overload
>>>>    (it gets message timeouts on larger templates). I wonder if this is the
>>>>    problem in our convergence gate...
>>>>
>>>> Good spotting. If it's true, probably we should try to change  number
>>> of engines... (not sure, how gate hardware react on it).
>>>
>>>>
>>>>    - convergence does very well with a reasonable number of engines
>>>>    running.
>>>>    - delete is slightly slower on convergence
>>>>
>>>>
>>> Also about delete - may be we may to optimize it later, when convergence
>>> way get more feedback.
>>>
>>>>
>>>> Still to test:
>>>>
>>>>    - the above, but measure memory usage
>>>>    - many small templates (run concurrently)
>>>>    - we need to ask projects using Heat to try with convergence
>>>>    (Murano, TripleO, Magnum, Sahara, etc..)
>>>>
>>>> Any feedback welcome (suggestions on what else to test).
>>>>
>>>> -Angus
>>>>
>>>>
>>>> __________________________________________________________________________
>>>> OpenStack Development Mailing List (not for usage questions)
>>>> Unsubscribe:
>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>
>>>
>>> Regards,
>>> Sergey.
>>>
>>>
>>>
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>>
>> --
>> Sincerely yours,
>> Sergey Lukjanov
>> Sahara Technical Lead
>> (OpenStack Data Processing)
>> Principal Software Engineer
>> Mirantis Inc.
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150831/54fe49b9/attachment.html>