Open Stack

> > But I guess you're suggesting not only that we version/schematize
> > individual notification payloads, but that we do so in a way that's
> > global across event types and emitters?
> 
> That's partially correct. I'm suggesting the we consider standardizing a
> general format for notification payloads across emitters. _Not_
> individual notification payloads.

So I'm open to correction, but I don't think anyone suggested using
egregiously different formats for different notifications.

A notification of compute.instance.create.start is naturally going to
carry different types of data than a volume.snapshot.delete.end for
example, but of course we'd seek to accommodate that difference within
a generic structure as far as possible.
 
> Schematizing individual payloads strikes me as heavyweight[1] when it
> _might_[2] be easier to declare that "a list of dicts with certain
> required fields" is sufficient to represent a multiplicity of value-
> oriented events which are consumed as any of metric, event,
> visualization data-point, etc.

OK, so lightweight schemas are lighter in weight that heavyweight
schemas. Yep, agreed.

> Elsewhere there's been some discussion of ActivityStreams as a
> possible model.

Is that the social media syndication format you're talking about?

That's been discussed elsewhere as possible model for openstack
notifications? I missed that, can you provide a link?

> It may be, they have the same basic idea: "stuff that
> happens" can be represented by a sequence of relatively stupid event-
> like things.

OK, I might be missing something here, but we seem to have a close
approximation of that already:

  stuff happens ==> events pop out on the bus

So is your point that our events aren't dumb enough?

(e.g. encode too much structure, or carry too much data, or require
too much interpretation?) 

> The intial versions of ActivityStreams was based on atom+xml but later
> people realized "this would be a ton easier if it was some well known
> and well formed JSON".

Are you suggesting here we adopt atom+json? 

> > So for the purposes of grounding the discussion, can you give an
> > example of what the Ironic notification payload might look like in
> > a perfect world?
> 
> I can't give you a perfect example because I haven't had an
> opportunity to understand all the needs and issues but a rough quickie
> to try to illustrate the point is here:
> 
>      http://paste.openstack.org/show/86071/
> 
> The main difference there is that the value and the unit are provided
> in known fields at a known descent into the data structure, rather than
> needing to be extracted from strings that are values at custom keys.

OK, so in this case, can we sum it up with:

 * provide an ID field

 * parse out the Sensor Reading into value and units, dropping the
   delta

 * push the concept of cumulative versus delta versus gauge into the
   producer

 * dump the stuff that ceilometer is not so interested into an 'extras;
   dict

So, cool, we've made our job a little easier on the ceilometer side.

Has the notification lost any expressive power? Not much by the looks
of it.

Is it a good idea to drop for example the error range on the floor
because ceilometer doesn't know what to do with it?

Dunno, that would depend on whether (a) the delta value reported by
IPMI is actually meaningful and (b) some other consumer can do something
smart with it.

I don't know enough about IPMI to comment on (a). My instinct would be
to leave the door open for (b).

But overall, it doesn't look like too radical a change in format. 

> I'd rather we not get hung up on the details of the representation as
> that's the way rabbits go. If the concept has merit the representation
> can follow.
> 
> >> Yes, this moves the onus of creating well-formed metrics to notifying
> >> systems, but that is good: It is providing exactly the sort of clean
> >> and easy to test contract at the boundaries that leads to good
> >> neighbors and easy testing.
> >
> > OK, so the thing to note is that the set of potential consumers is
> > not limited to services with a metrics-oriented view of the world.
> 
> I'm of the opinion that it _may_ be possible that "metrics-oriented"
> is a subclass of something more generic that might be representable.
> The CADF folk seem to think so (although I'm not suggesting we go down
> that road, c.f. "heavyweight").

And again, unless I'm missing something, that's pretty much what I
was saying. Re-using your phraseology:

  "creating well-formed metrics" is not necessarily what we want the
  producer to do, because "metrics-oriented is a subclass of something
  more generic"
 
> > * services with a metrics-oriented PoV (e.g. Ceilometer)
> > * services with an events-oriented PoV (e.g. StackTach)
> > * services with a UX-oriented PoV (e.g. Horizon)
> > * services with an entitlement-enforcement PoV (e.g. Katello)
> 
> These all sound like there are in at least the same ballbark. I think
> the common term would be "events" and a metric is a type of event?

Yes. Yes. And thrice yes! :)

The metric-oriented view is subset/specialized aspect of something
more generic. 

> > But in an case, I'm bit confused by where this line of reasoning is
> > headed. We seem to have gone from the notifying service directly
> > emitting well-formed samples that ceilometer can just directly
> > persist, to more generic "event-like thingies".
> >
> > Have I misread that point about *directly* emitting samples?
> 
> No, you are seeing the evolution of my thinking: as I write about this
> stuff and try to understand it, it becomes more clear (to me) that
> samples ought to be treated as event-like thingies.

OK, so that shift makes it a little difficult to follow and engage with.

> >> Having a standard notification payload format would of course mean change,
> >> but we know that flexible metering/auditing is very important for the
> >> OpenStack universe. Your argument seems to be that having such a
> >> standard, predisposed to ceilometer, would limit flexibility and lose
> >> capability.
> >
> > Yep, that is exactly my point. Predisposition of the notification format
> > to ceilometer's needs is what concerns me. As opposed to the notion of
> > standardization/schematization/versioning which is the explicit goal of
> > the discussion.
> 
> Okay, and my proposal is not to have a standard predisposed to
> ceilometer. It is to make a limited standard for a large class of
> notifications and _not_ schematize individual notification types (more
> on this aspect below) and then change ceilometer so it can use the plan.

OK, so perhaps I misread your earlier comments on samples and assumed
a predisposition to ceilometer's requirements was being proposed?

Can you start fleshing out exactly what you mean by a standard not
necessarily predisposed to ceilometer, still sufficiently close to
eliminate the need for a mapping layer on the ceilometer side?

And then please run this by the other stake-holders (e.g. StackTach)
for a sanity check to confirm that it would complicate their lives
unnecessarily.

> >> Take my example from above, the processing of Ironic notifications.
> >> I think it is weird that I had to write code for that. Does it not
> >> seem odd to you?
> >
> > OK, so have we strayed into an orthogonal concern here?
> 
> I don't think so, it was supposed to be an example of the cost of there
> not being an existing general standard. If there were such a standard I
> wouldn't have had to write any code, only the Ironic folk would and I
> would have had the free time to help them. Less code == good!

Yes less code is great as long as we don't sacrifice the flexibility
that we and other consumers need.

> Similarly if there are individual schema for the various notification
> types, every time someone wants to make a new notification they need
> to get that schema validated and various agents and actors need to be
> informed of its existence and then the new thing needs to be
> integrated. That is limiting. If the contract can be managed at a higher
> layer of abstraction more agents and actors can contribute.

I don't really know what a "higher layer of abstraction" actually means
when we talk about schema. More generic, less specific? Something else?

> > i.e. not that ceilometer requires some translation to be done, but
> > that this translation must be hand-craft in Python code as opposed
> > to being driven declaratively via some configured mapping rules?
> 
> I don't think there needs to be much, if any, mapping on the consumption
> side of the notification process if there is a standard form in which
> those notifications are emitted. In those cases where pipeline
> transformation needs to be done ( multiple value gathering) the pipeline
> can consume certain notifications and then emit more as the result of
> the transformation.

OK just being devil's advocate for a second, it sounds like a truism
to state we wouldn't need mapping logic if the mapping was done elsewhere.
The point is to map at the appropriate points in time, without shoehorning
stuff into a representation that makes life more difficult for other
consumers.

If you can figure out how to do that in a concrete way ... then great,
I'd be receptive, let's hear some proposals.

Cheers,
Eoghan