Open Stack

On Thu, Apr 9, 2015 at 2:15 AM, Clint Byrum <clint at fewbar.com> wrote:

> Excerpts from Angus Salkeld's message of 2015-04-06 19:55:37 -0700:
> > Hi all
> >
> > For quite some time we (Heat team) have wanted to be able to send
> messages
> > to our
> > users (by user I do not mean the Operator, but the User that is
> interacting
> > with the client).
> >
> > What do I mean by "user messages", and how do they differ from our
> current
> > log messages
> > and notifications?
> > - Our current logs are for the operator and have information that the
> user
> > should not have
> >   (ip addresses, hostnames, configuration options, other tenant info
> etc..)
> > - Our notifications (that Ceilometer uses) *could* be used, but I am not
> > sure if it quite fits.
> >   (they seem a bit heavy weight for a log message and aimed at higher
> level
> > events)
> >
> > These messages could be (based on Heat's use case):
> >
> > - Specific user oriented log messages (distinct from our normal operator
> > logs)
>
> These currently go in the Heat events API, yes?
>

Well I wanted a replacement for our events table that could also take
smaller messages
and fulfil some other use cases (be a stream, that the user doesn't have to
poll).

We are sending RPC notifications at the moment:
stack.<action>.begin/end/error
autoscaling.begin/end/error

We need to add:
per resource notifications (to replace the event table)

Add some useful log-like messages:
"x property is deprecated, please update your template to use Y"

There are also a bunch of other log messages that should really be going to
the user, but
just end up in the Operator's log file.


> > - Deprecation messages (if they are using old resource
> properties/template
> > features)
>
> I think this could fit with the bits above.
>
> > - Progress and resource state changes (an application doesn't want to
> poll
> > an api for a state change)
>
> These also go in the current Heat events.
>
> > - Automated actions (autoscaling events, time based actions)
>
> As do these?
>
> > - Potentially integrated server logs (from in guest agents)
> >
> > I wanted to raise this to "[all]" as it would be great to have a general
> > solution that
> > all projects can make use of.
> >
> > What do we have now:
> > - The user can not get any kind of log message from services. The closest
> > thing
> >   ATM is the notifications in Ceilometer, but I have the feeling that
> these
> > have a different aim.
> > - nova console log
> > - Heat has a DB "event" table for users (we have long wanted to get rid
> of
> > this)
>
> So if we forget the DB part of it, the API is also lacking things like
> pagination and search that one would want in an event/logging API.
>

I'd almost rather spend the time getting an OpenStack wide solution then
switch
our client (and API) to use that.


>
> >
> > What do other clouds provide:
> > - https://devcenter.heroku.com/articles/logging
> > - https://cloud.google.com/logging/docs/
> > - https://aws.amazon.com/blogs/aws/cloudwatch-log-service/
> > - http://aws.amazon.com/cloudtrail/
> > (other examples...)
> >
> > What are some options we could investigate:
> > 1. remote syslog
> >     The user provides a rsyslog server IP/port and we send their messages
> > to that.
> >     [pros] simple, and the user could also send their server's log
> messages
> > to the same
> >               rsyslog - great visibility into what is going on.
> >
> >               There are great tools like loggly/logstash/papertrailapp
> that
> > source logs from remote syslog
> >               It leaves the user in control of what tools they get to
> use.
> >
> >     [cons] Would we become a spam agent (just sending traffic to an
> > IP/Port) - I guess that's how remote syslog
> >                works. I am not sure if this is an issue or not?
> >
> >               This might be a lesser solution for the use case of "an
> > application doesn't want to poll an api for a state change"
> >
> >               I am not sure how we would integrate this with horizon.
> >
>
> I think this one puts too much burden on the user to setup a good
> receiver.
>
> > 2. Zaqar
> >     We send the messages to a queue in Zaqar.
> >     [pros] multi tenant OpenStack project for messaging!
> >
> >     [cons] I don't think Zaqar is installed in most installations (tho'
> > please correct me here if this
> >                is wrong). I know Mirantis does not currently support
> Zaqar,
> > so that would be a problem for me.
> >
> >               There is not the level of external tooling like in option
> "1"
> > (logstash and friends)
> >
>
> I agree with your con, and would also add that after the long
> discussions we had in the past we had some concerns about scaling.
>
> > 3. Other options:
> >    Please chip in with suggestions/links!
> >
>
> There's this:
>
> https://wiki.openstack.org/wiki/Cue
>
> I think that could be a bit like 1, but provide the user with an easy
> target for the messages.
>
> I also want to point out that what I'd actually rather see is that all
> of the services provide functionality like this. Users would be served
> by having an event stream from Nova telling them when their instances
> are active, deleted, stopped, started, error, etc.
>
> Also, I really liked Sandy's suggestion to use the notifications on the
> backend, and then funnel them into something that the user can consume.
> The project they have, yagi, for putting them into atom feeds is pretty
> interesting. If we could give people a simple API that says "subscribe
> to Nova/Cinder/Heat/etc. notifications for instance X, and put them
> in an atom feed", that seems like something that would make sense as
> an under-the-cloud service that would be relatively low cost and would
> ultimately reduce load on API servers.
>

"an under-the-clould service" ? - That is not what I am after here.


What I am really after is a general OpenStack solution for how end users can
consume service notifications (and replace "heat event-list").


Right now there is "ceilometer event-list", but as some Ceilometer devs
have said,
they don't want to store every notification that comes.

So is the yagi + atom hopper solution something we can point end-users to?
Is it per-tenant etc...

Sandy, do you have a write up somewhere on how to set this up so I can
experiment a bit?

Maybe this needs to be a part of Cue?

-Angus


>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150409/ce1ed08b/attachment.html>