Open Stack

> - Specific user oriented log messages (distinct from our normal operator logs)> - Deprecation messages (if they are using old resource properties/template features)> - Progress and resource state changes (an application doesn't want to poll an api for a state change)
> - Automated actions (autoscaling events, time based actions)> - Potentially integrated server logs (from in guest agents)
is the idea that Heat would build events from the logs or would you want to send the log messages to another service to be process? so for example, Nova doesn't send all logs messages to the queue but they do send a set of messages relating to certain actions and errors that occur (beyond just CRUD events).  as the use cases above seem to target specific actions/logs and not all logs, i would think the processing could be done on the initiators service end and not on the consumer end.

to give an example of what Ceilometer is capable of; Ceilometer currently takes JSON messages from the MQ from *most* services and from there we capture the entire raw notification and index on a select set of key-value pairs. i think it's entirely possible to take in non-json log messages and build an indexer around that if needed.

cheers,
gord


Date: Tue, 7 Apr 2015 12:55:37 +1000
From: asalkeld at mirantis.com
To: openstack-dev at lists.openstack.org
Subject: [openstack-dev] [all] how to send messages (and events) to our users

Hi all
For quite some time we (Heat team) have wanted to be able to send messages to ourusers (by user I do not mean the Operator, but the User that is interacting with the client).
What do I mean by "user messages", and how do they differ from our current log messagesand notifications?- Our current logs are for the operator and have information that the user should not have  (ip addresses, hostnames, configuration options, other tenant info etc..)- Our notifications (that Ceilometer uses) *could* be used, but I am not sure if it quite fits.   (they seem a bit heavy weight for a log message and aimed at higher level events)
These messages could be (based on Heat's use case):
- Specific user oriented log messages (distinct from our normal operator logs)- Deprecation messages (if they are using old resource properties/template features)- Progress and resource state changes (an application doesn't want to poll an api for a state change)
- Automated actions (autoscaling events, time based actions)- Potentially integrated server logs (from in guest agents)
I wanted to raise this to "[all]" as it would be great to have a general solution that all projects can make use of.
What do we have now:
- The user can not get any kind of log message from services. The closest thing  ATM is the notifications in Ceilometer, but I have the feeling that these have a different aim.- nova console log- Heat has a DB "event" table for users (we have long wanted to get rid of this)
What do other clouds provide:- https://devcenter.heroku.com/articles/logging- https://cloud.google.com/logging/docs/- https://aws.amazon.com/blogs/aws/cloudwatch-log-service/- http://aws.amazon.com/cloudtrail/(other examples...)
What are some options we could investigate:1. remote syslog    The user provides a rsyslog server IP/port and we send their messages to that.    [pros] simple, and the user could also send their server's log messages to the same              rsyslog - great visibility into what is going on.
              There are great tools like loggly/logstash/papertrailapp that source logs from remote syslog              It leaves the user in control of what tools they get to use.
    [cons] Would we become a spam agent (just sending traffic to an IP/Port) - I guess that's how remote syslog               works. I am not sure if this is an issue or not?
              This might be a lesser solution for the use case of "an application doesn't want to poll an api for a state change"
              I am not sure how we would integrate this with horizon.
2. Zaqar    We send the messages to a queue in Zaqar.    [pros] multi tenant OpenStack project for messaging!
    [cons] I don't think Zaqar is installed in most installations (tho' please correct me here if this               is wrong). I know Mirantis does not currently support Zaqar, so that would be a problem for me.
              There is not the level of external tooling like in option "1" (logstash and friends)
3. Other options:   Please chip in with suggestions/links!   

https://blueprints.launchpad.net/heat/+spec/user-visible-logs


RegardsAngus




__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150407/8515cac0/attachment.html>