Open Stack

Happens all of the time. I haven't been able to get a single meter stored. :(

From: Stas Maksimov <maksimov at gmail.com<mailto:maksimov at gmail.com>>
Reply-To: OpenStack Development Mailing List <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Date: Monday, August 12, 2013 9:34 AM
To: OpenStack Development Mailing List <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Subject: Re: [openstack-dev] [Ceilometer] Need help with HBase backend

Is it sporadic or happens all the time?

In my case my Ceilometer VM was different from HBase VM, so I'm not sure if DHCP issues can affect localhost connections.

Thanks,
Stas

On 12 August 2013 15:29, Thomas Maddox <thomas.maddox at rackspace.com<mailto:thomas.maddox at rackspace.com>> wrote:
Hmmm, that's interesting.

That would effect an all-in-one deployment? It's referencing localhost right now; not distributed. My Thrift server is hbase://127.0.0.1:9090/<http://127.0.0.1:9090/>. Or would that still effect it, because it's a software facilitated localhost reference and I'm doing dev inside of a VM (in the cloud) rather than a hardware host?

I really appreciate your help!

-Thomas

From: Stas Maksimov <maksimov at gmail.com<mailto:maksimov at gmail.com>>
Reply-To: OpenStack Development Mailing List <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Date: Monday, August 12, 2013 9:17 AM
To: OpenStack Development Mailing List <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Subject: Re: [openstack-dev] [Ceilometer] Need help with HBase backend

Aha, so here it goes. The problem was not caused by monkey-patching or multithreading issues, it was caused by the DevStack VM losing its connection and getting a new address from the DHCP server. Once I fixed the connection issues, the problem with eventlet disappeared.

Hope this helps,
Stas

On 12 August 2013 14:49, Stas Maksimov <maksimov at gmail.com<mailto:maksimov at gmail.com>> wrote:

Hi Thomas,

I definitely saw this before, iirc it was caused by monkey-patching somewhere else in ceilometer. It was fixed in the end before i submitted hbase implementation.

At this moment unfortunately that's all I can recollect on the subject. I'll get back to you if I have an 'aha' moment on this. Feel free to contact me off-list regarding this hbase driver.

Thanks,
Stas.

Hey team,

I am working on a fix for retrieving the latest metadata on a resource rather than the first with the HBase implementation, and I'm running into some trouble when trying to get my dev environment to work with HBase. It looks like a concurrency issue when it tries to store the metering data. I'm getting the following error in my logs (summary):

013-08-11 18:52:33.980 2445 ERROR ceilometer.collector.dispatcher.database [req-3b3c65c9-1a1b-4b5d-bba5-8224f074b176 None None] Second simultaneous read on fileno 7 detected.  Unless you really know what you're doing, make sure that only one greenthread can read any particular socket.  Consider using a pools.Pool. If you do know what you're doing and want to disable this error, call eventlet.debug.hub_prevent_multiple_readers(False)

Full traceback: http://paste.openstack.org/show/43872/

Has anyone else run into this lovely little problem? It looks like the implementation needs to use happybase.ConnectionPool, unless I'm missing something.

Thanks in advance for help! :)

-Thomas

_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130812/0e0a7a60/attachment.html>