Open Stack

Hi guys,

I decided to forward this message to dev list too. On the previous week I
was investigating Ceilometer performance. And this letter is a brief
description of my results.

Lab description:
3 controllers
187 computes
HA: Galera for MySQL
memcached is on, RabbitMQ in HA mode

Ceilometer processes are running as follows:
1 controller: ceilometer-api, ceilometer-agent-central,
ceilometer-collector, ceilometer-agent-compute
2 and 3 controller: ceilometer-collector
all computes: ceilometer-agent-compute

My idea was just to see how Ceilometer works in real-life circumstances. I
decided to use Rally to see how much Ceilometer's work affects the speed of
instance booting and removing time. Rally is a new instrument for me and I
hope Rally's guys will fix me if I'm wrong in config description.

Scenario:
1. Test boot-time without Ceilometer. Just an empty Lab
2. Run manually 100 instances which will be always polled by Ceilometer in
next scenarios
3. Start Ceilometer with different "interval" in pipeline.conf in _all_
computes and controllers.
4. Use the following Rally config:
"NovaServers.boot_and_delete_server": [
        {"args": {"flavor_id": 2, "image_id":
"430c55c4-d9f6-4fa9-9888-5ca1f9531530"},
         "execution": "continuous",
         "config": {"times": 400, "active_users": 100, "tenants": 10,
"users_per_tenant": 20}}
      ]
The config means that the method boot_and_delete_server (create one
instance and delete it at once) will be called 400 times. Max "level of
concurrency" is 100 i.e. 100 active users (as max) may create instance at
the same time. Amount of users = tenants*users_per_tenant = 200.

Rally measures how long method 'boot_and_delete_server' worked. And here is
a list of results:

+----------------------+---------------------+---------------------+--------------------+
|      max               |      avg              |      min               |
scenario           |
+----------------------+---------------------+---------------------+--------------------+
| 288.868812084 | 109.638720818 | 35.1642079353 |   empty lab       |
+---------------------+----------------------+---------------------+--------------------+
| 321.71191287   | 173.705494534 | 72.4606912136 |  20 sec polling  |
+---------------------+----------------------+---------------------+--------------------+
| 389.527067184 | 217.689281176 | 71.4955689907 | 10 sec polling  |
+---------------------+----------------------+---------------------+--------------------+
| 445.328452826 | 243.213246702 | 117.31561017   |  5 sec polling   |
+---------------------+----------------------+---------------------+--------------------+

The second interesting moment is that Ceilometer affects user creation very
much. Unfortunately, Rally doesn't measure this metric and I cannot provide
concrete results.

Guys, if you have any questions or comments you are welcome! I think that
2x difference between avg time in "empty lab" and "5 sec polling" scenario
is not a bad result. But 100 instances that were being monitored during the
test is not a real load for the lab. What do you think? Should I repeat the
test with 1000 instances?

Thanks,
Nadya
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131210/8f3e69ca/attachment.html>