Open Stack

Notifications were added for this very purpose.


At Rax, we have a downstream consumer (yagi) that routes notifications to an appropriate ATOM/pubsub feed (based on tenant-id).


Notifications are only as heavy as you want to make them. The only required fields are event_type and timestamp. ?


________________________________
From: Angus Salkeld <asalkeld at mirantis.com>
Sent: Monday, April 6, 2015 11:55 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [all] how to send messages (and events) to our users

Hi all

For quite some time we (Heat team) have wanted to be able to send messages to our
users (by user I do not mean the Operator, but the User that is interacting with the client).

What do I mean by "user messages", and how do they differ from our current log messages
and notifications?
- Our current logs are for the operator and have information that the user should not have
  (ip addresses, hostnames, configuration options, other tenant info etc..)
- Our notifications (that Ceilometer uses) *could* be used, but I am not sure if it quite fits.
  (they seem a bit heavy weight for a log message and aimed at higher level events)

These messages could be (based on Heat's use case):

- Specific user oriented log messages (distinct from our normal operator logs)
- Deprecation messages (if they are using old resource properties/template features)
- Progress and resource state changes (an application doesn't want to poll an api for a state change)
- Automated actions (autoscaling events, time based actions)
- Potentially integrated server logs (from in guest agents)

I wanted to raise this to "[all]" as it would be great to have a general solution that
all projects can make use of.

What do we have now:
- The user can not get any kind of log message from services. The closest thing
  ATM is the notifications in Ceilometer, but I have the feeling that these have a different aim.
- nova console log
- Heat has a DB "event" table for users (we have long wanted to get rid of this)

What do other clouds provide:
- https://devcenter.heroku.com/articles/logging
- https://cloud.google.com/logging/docs/
- https://aws.amazon.com/blogs/aws/cloudwatch-log-service/
- http://aws.amazon.com/cloudtrail/
(other examples...)

What are some options we could investigate:
1. remote syslog
    The user provides a rsyslog server IP/port and we send their messages to that.
    [pros] simple, and the user could also send their server's log messages to the same
              rsyslog - great visibility into what is going on.

              There are great tools like loggly/logstash/papertrailapp that source logs from remote syslog
              It leaves the user in control of what tools they get to use.

    [cons] Would we become a spam agent (just sending traffic to an IP/Port) - I guess that's how remote syslog
               works. I am not sure if this is an issue or not?

              This might be a lesser solution for the use case of "an application doesn't want to poll an api for a state change"

              I am not sure how we would integrate this with horizon.

2. Zaqar
    We send the messages to a queue in Zaqar.
    [pros] multi tenant OpenStack project for messaging!

    [cons] I don't think Zaqar is installed in most installations (tho' please correct me here if this
               is wrong). I know Mirantis does not currently support Zaqar, so that would be a problem for me.

              There is not the level of external tooling like in option "1" (logstash and friends)

3. Other options:
   Please chip in with suggestions/links!


https://blueprints.launchpad.net/heat/+spec/user-visible-logs


Regards
Angus



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150407/83724901/attachment.html>