Open Stack

On 05/26/2016 02:11 PM, Matthew Treinish wrote:
> On Thu, May 26, 2016 at 05:55:34PM +0000, Jeremy Stanley wrote:
>> On 2016-05-26 12:54:49 -0400 (-0400), Matthew Treinish wrote:
>>> Just a quick follow-up I started running this on a throwaway server at
>>> 15.184.138.236. So you can subscribe to events from gerrit.
>> [...]
>>
>> How resource-intensive is it? Curious whether it makes sense to run
>> something like this directly on review.openstack.org. If zuul grew
>> support for that mechanism, it might allow CI systems (third party
>> or even our own) to wean off using SSH entirely since this is a
>> problem in a lot of places from crazy enterprise firewall policies
>> to systems running in mainland China.

It does run on a custom port... so the great firewall of China is still
probably an issue.

> It's eating like nothing on my server right now. This is all running on a single
> cpu vm on a private cloud with a "Intel Xeon E312xx (Sandy Bridge)" (according
> to /proc/cpuinfo) Mosquitto itself has a memory footprint < 1MB and I've seen it
> spike up to a whopping 1% cpu utilization. Although, this might increase a as more
> subscribers are added. This is the first time I've played with mosquitto and mqtt
> so I don't know what it's scaling is like. But, I imagine it should handle a lot
> of subscriptions well since it's supposed to be an IoT thing. germqtt is eating
> a bit more with consuming about 1.5M of RAM and it uses about the same CPU as
> Mosquitto.

In my ideal world of awesomeness, there would be an MQTT server in infra
which was getting data from all the relevant change sources (gerrit
changes, launchpad changes (which, yes requires something crazy like
converting an email stream into events)), zuul enqueue, dequeue events).

And then every time someone wanted to build some ad hoc web tool to take
a slice of this they could consume the event stream for updates, instead
of doing what everyone does and polls once an hour. As long as the topic
trees are well structured, it should make for an easy way to get the
slice they needed and only react to that.

	-Sean

-- 
Sean Dague
http://dague.net