Open Stack

apologies, if the below was mentioned at some point in this thread.

On 04/11/2015 10:42 AM, Sean Dague wrote:
> This seems like a fundamental abuse of HTTP honestly. If you find
> yourself creating a ton of new headers, you are probably doing it wrong.
if we want to explore the HTTP path, did we consider using ETags[1] to 
check whether resources have changed? it's something used by Gnocchi's 
API to handle resource changes.
>
> I do think the near term work around is to actually use Searchlight.
> They're monitoring the notifications bus for nova, and refreshing
> resources when they see a notification which might have changed it. It
> still means that Searchlight is hitting our API more than ideal, but at
> least only one service is doing so, and if the rest hit that instead
> they'll get the resource without any db hits (it's all through an
> elastic search cluster).
>
> I think longer term we probably need a dedicated event service in
> OpenStack. A few of us actually had an informal conversation about this
> during the Nova notifications session to figure out if there was a way
> to optimize the Searchlight path. Nearly everyone wants websockets,
> which is good. The problem is, that means you've got to anticipate
> 10,000+ open websockets as soon as we expose this. Which means the stack
> to deliver that sanely isn't just a bit of python code, it's also the
> highly optimized server underneath.
as part of the StackTach integration efforts, Ceilometer (as of Juno) 
listens to all notifications in the OpenStack ecosystem and builds a 
normalised event model[2] from it. the normalised event data is stored 
in a backend (elasticsearch, sql, mongodb, hbase) and from this you can 
query based on required attributes. in addition to storing events, in 
Liberty, Aodh (alarming service) added support to take events and create 
alarms based on change of state[3] with expanded functionality to be 
added. this was added to handle the NFV use case but may also be 
relevant here as it seems like we want to have an action based on status 
changes.

i should mention that we discussed splitting out the event logic in 
Ceilometer to create a generic listener[4] service which could convert 
notification data to meters, events, and anything else. this isn't a 
high priority item but might be an integration point for those looking 
to leverage notifications in OpenStack.

[1] https://en.wikipedia.org/wiki/HTTP_ETag
[2] http://docs.openstack.org/admin-guide-cloud/telemetry-events.html
[3] 
http://specs.openstack.org/openstack/ceilometer-specs/specs/liberty/event-alarm-evaluator.html
[4] https://etherpad.openstack.org/p/mitaka-telemetry-split

cheers,

-- 
gord