Open Stack

On Thu, Apr 3, 2014 at 6:24 AM, Yuanjing (D) <yj.yuan at huawei.com> wrote:
> Hi:
>
>
>
> I have deployed a RDO havana version's openstack environment for a period.
>
>
>
> By chance I tested APIs of ceilometer, in the first I defined a new meter
> like meter_test, then used 'ceilometer sample-list -m yjmeter' command to
> check the result.
>
> I found that the query cost about 13 seconds, why it was so slow with one
> record?
>
>
>
> Now that RDO used mongodb as default DB for ceilometer, I logging into it
> and found that the meter collection had about 2,700,000 records. Maybe the
> mass records slowed the query.
>
>
>
> Take production environemnt into account, If there are 100 hosts , each host
> contains 30 VM.
>
> CeilometerI use default configuration of 11 compute pollsters and the
> pipeline interval is 600s.
>
> Then one day system will generate 4,752,000 record, the formula is:
> 11(meter) * 30(VM) * 6(system run pollsters 6 times an hour) * 24(a day) *
> 100(host) = 4,752,000 records.
>
>
>
> For above case I think it is necessary to restrict number of records by some
> mechanism, below are my immature idea:
>
> 1. Make restriction on max supported records by time or number of samples,
> discard old records.

There is a mechanism to delete old data in place using an age.

>
> 2. Providing API to delete samples by resource_id or some other conditions,
> so the third integration system may call this API to delete related samples
> when delete a resource.

That's not a bad idea, although you wouldn't always want to call it.
Metering data should live longer than the resource being billed.

>
> 3. Running period task of accounting on raw samples, using 1min samples to
> generate 5min statistics samples, using 5min statistics to generate 30min
> statistics samples, and so on. Every period of sample has individual data
> table and has resriction on max supported records .

This sort of roll-up would be useful for monitoring, but would break
the audit trail for metering. So it might be useful, but may not solve
the whole problem.

> I am not a ceilometer programmer and I apologize if I am missing something
> very obvious.

The rate of collection, meters kept, and retention time are all
configurable using the main configuration file or the pipeline YAML
file. That's not to say this isn't a real problem -- I see one or two
summit sessions that may cover performance issues like this, and I
know it is a high priority for both of our PTL candidates.

Doug

>
> Can you give me some help to make me clear about them and how to implement
> my requirement?
>
>
>
> Thanks
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>