Open Stack

@Maish Saidel-Keesing,

Hi Maish, I’m from eBay Inc, and we’re enabling 1000+ ceilometer compute agents. Hope our experience could help.

We choose an OpenTSDB backend instead of MongoDB in the first place, so we avoid of most of the issues related to MongoDB.

However, during deployment, we still met many issues as below:

  1.  The inspector of libvirt didn’t work in nova-cell mode. We fixed it by using instance uuid to identify vm, and submitted to upstream. (https://bugs.launchpad.net/ceilometer/+bug/1396473)
  2.  There’s huge load to nova/glance client that even drag them down. We resolved it in 3 ways as below to reduce the load:
     *   Shuffle compute agents triggering time to avoid same time requests to nova client., and it’s already got approved and merged in upstream. (https://bugs.launchpad.net/ceilometer/+bug/1412613)
     *   Add cache layer for nova discovery results of instances, this would reduce quite a lot queries to nova client. It’s still in discussing with upstream. (https://review.openstack.org/#/c/153503/)
     *   Remove flavor and image query for vm since we didn't need the info now.

Our original thinking about MongoDB is to only store some metadata definition, and put most other metrics to a time series db.

So all in all, we think probably you can consider to change your main storage backend MongoDB, and that may improve your Ceilometer performance.
Also some performance related code enhance/modification based on your conditions would be better.

Thanks,
Bryant(Cloud Team, eBay Inc)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20150212/43571ca0/attachment.html>