Open Stack

Excerpts from Angus Salkeld's message of 2015-04-06 19:55:37 -0700:
> Hi all
> 
> For quite some time we (Heat team) have wanted to be able to send messages
> to our
> users (by user I do not mean the Operator, but the User that is interacting
> with the client).
> 
> What do I mean by "user messages", and how do they differ from our current
> log messages
> and notifications?
> - Our current logs are for the operator and have information that the user
> should not have
>   (ip addresses, hostnames, configuration options, other tenant info etc..)
> - Our notifications (that Ceilometer uses) *could* be used, but I am not
> sure if it quite fits.
>   (they seem a bit heavy weight for a log message and aimed at higher level
> events)
> 
> These messages could be (based on Heat's use case):
> 
> - Specific user oriented log messages (distinct from our normal operator
> logs)

These currently go in the Heat events API, yes?

> - Deprecation messages (if they are using old resource properties/template
> features)

I think this could fit with the bits above.

> - Progress and resource state changes (an application doesn't want to poll
> an api for a state change)

These also go in the current Heat events.

> - Automated actions (autoscaling events, time based actions)

As do these?

> - Potentially integrated server logs (from in guest agents)
> 
> I wanted to raise this to "[all]" as it would be great to have a general
> solution that
> all projects can make use of.
> 
> What do we have now:
> - The user can not get any kind of log message from services. The closest
> thing
>   ATM is the notifications in Ceilometer, but I have the feeling that these
> have a different aim.
> - nova console log
> - Heat has a DB "event" table for users (we have long wanted to get rid of
> this)

So if we forget the DB part of it, the API is also lacking things like
pagination and search that one would want in an event/logging API.

> 
> What do other clouds provide:
> - https://devcenter.heroku.com/articles/logging
> - https://cloud.google.com/logging/docs/
> - https://aws.amazon.com/blogs/aws/cloudwatch-log-service/
> - http://aws.amazon.com/cloudtrail/
> (other examples...)
> 
> What are some options we could investigate:
> 1. remote syslog
>     The user provides a rsyslog server IP/port and we send their messages
> to that.
>     [pros] simple, and the user could also send their server's log messages
> to the same
>               rsyslog - great visibility into what is going on.
> 
>               There are great tools like loggly/logstash/papertrailapp that
> source logs from remote syslog
>               It leaves the user in control of what tools they get to use.
> 
>     [cons] Would we become a spam agent (just sending traffic to an
> IP/Port) - I guess that's how remote syslog
>                works. I am not sure if this is an issue or not?
> 
>               This might be a lesser solution for the use case of "an
> application doesn't want to poll an api for a state change"
> 
>               I am not sure how we would integrate this with horizon.
> 

I think this one puts too much burden on the user to setup a good
receiver.

> 2. Zaqar
>     We send the messages to a queue in Zaqar.
>     [pros] multi tenant OpenStack project for messaging!
> 
>     [cons] I don't think Zaqar is installed in most installations (tho'
> please correct me here if this
>                is wrong). I know Mirantis does not currently support Zaqar,
> so that would be a problem for me.
> 
>               There is not the level of external tooling like in option "1"
> (logstash and friends)
>

I agree with your con, and would also add that after the long
discussions we had in the past we had some concerns about scaling.

> 3. Other options:
>    Please chip in with suggestions/links!
> 

There's this:

https://wiki.openstack.org/wiki/Cue

I think that could be a bit like 1, but provide the user with an easy
target for the messages.

I also want to point out that what I'd actually rather see is that all
of the services provide functionality like this. Users would be served
by having an event stream from Nova telling them when their instances
are active, deleted, stopped, started, error, etc.

Also, I really liked Sandy's suggestion to use the notifications on the
backend, and then funnel them into something that the user can consume.
The project they have, yagi, for putting them into atom feeds is pretty
interesting. If we could give people a simple API that says "subscribe
to Nova/Cinder/Heat/etc. notifications for instance X, and put them
in an atom feed", that seems like something that would make sense as
an under-the-cloud service that would be relatively low cost and would
ultimately reduce load on API servers.