Open Stack

> > A notification of compute.instance.create.start is naturally going to
> > carry different types of data than a volume.snapshot.delete.end for
> > example, but of course we'd seek to accommodate that difference within
> > a generic structure as far as possible.
> 
> Is it going to carry different types of data or different values on
> identical keys. The goal would be that the "meaning" of a notification
> for compute.instance.create.start and volume.snapshot.delete.end
> should be found on the same set of keys. That's all "the proposal" is.

OK, got it. :)

So what we need to figure out is how exactly this common structure can be
accommodated without reverting back to what Sandy called the "wild west"
in another post.

(i.e. the current scenario of a small set of common fields such as priority
or timestamp, followed by a free-form payload dict that differs from one
event type to another, or may even change from the old form of one event
type to the new form of the same notification)

> >> It may be, they have the same basic idea: "stuff that
> >> happens" can be represented by a sequence of relatively stupid event-
> >> like things.
> >
> > OK, I might be missing something here, but we seem to have a close
> > approximation of that already:
> >
> >  stuff happens ==> events pop out on the bus
> >
> > So is your point that our events aren't dumb enough?
> 
> Events are too different in from, from event to event.
> 
> > (e.g. encode too much structure, or carry too much data, or require
> > too much interpretation?)
> 
> Of these three, "require too much interpretation" is perhaps the best
> match.

OK, I think I can agree with the spirit of that.
 
> > But overall, it doesn't look like too radical a change in format.
> 
> No, it's not. What it is is a minor change that "everyone" could adopt
> and thus be able to play in the party.

Agreed, so I think we just need to tie down the benefits of such a non-
radical change, and compare those to the potential costs and benefits of
more locked-down schemata (i.e. more specific to individual event types).

> > The metric-oriented view is subset/specialized aspect of something
> > more generic.
> 
> Okay, so we agree on that. Can we agree that it might be possible to
> represent that generic thing with a common grammar?

Yes, I think we can :)
 
> >> No, you are seeing the evolution of my thinking: as I write about this
> >> stuff and try to understand it, it becomes more clear (to me) that
> >> samples ought to be treated as event-like thingies.
> >
> > OK, so that shift makes it a little difficult to follow and engage with.
> 
> This is a mailing list for a distributed project, it is the sole place
> where we can get some reasonable simulation of "having a chat" with a
> wide body of interested parties. The point of having a chat is to
> explore, think, and exchange and hone ideas. I certainly hope you
> don't expect me and everyone else only to write to the list when we
> have a fully researched proposal. This (apart from summits and
> mid-cycles) is the most accessible place we have to which we can go to
> create the information that allows us to do the compare and contrast
> the eventually leads to creating proposals.

Fully agreed that the ML is exactly the place for discussing and honing
ideas, outside of the design summit and mid-cycle meetups. And I agree
that there is no requirement for these ideas to be fully formed before
bringing them to the list. 

> > OK, so perhaps I misread your earlier comments on samples and assumed
> > a predisposition to ceilometer's requirements was being proposed?
> 
> It seems that way. Throughout I've been saying notifications should be
> easy for anyone to create and anyone to consume and have confidence
> they are doing it right.

Yep ... after discussing this on IRC, it seems that the confusion arose
from me reading too much into your prior statement of:

  "to me ceilometer is a service for gathering metrics and then being able
   to do things with those metrics. anything with the proper creds ought to
   be able to send out a notification which is to be treated as a metric
   and ceilometer should consume it"

i.e. assuming this to indicate a stronger equivalence between notifications
and metrics/samples than you had intended.
 
> > Can you start fleshing out exactly what you mean by a standard not
> > necessarily predisposed to ceilometer, still sufficiently close to
> > eliminate the need for a mapping layer on the ceilometer side?
> 
> That's what I'm trying to do here in this discussion here on this list
> because as far as I'm able to tell that's the one way to ensure that
> the idea has some validity and I'm not missing important details. I
> thought I was saying "how about this general idea, any merit? Help me
> flesh it out?" but apparently it came across as something different.

Apologies, I should have wrote: can you *continue* fleshing out ...

Poor choice of words on my part.

> That's a shame. If I were made of less stern stuff I might be put off
> trying to share ideas and make improvements: the hoops I'm leaping through
> are making me a feel bit "whoa, man, really I don't want to have to try
> this hard just to discuss an idea".

Discussion of ideas is definitely encouraged, and I would hate to think
that any other impression was given on this thread. So please do continue
developing your ideas here.

> > And then please run this by the other stake-holders (e.g. StackTach)
> > for a sanity check to confirm that it would complicate their lives
> > unnecessarily.
> 
> I would have thought that was what I was already doing by posting in
> this thread based on the title and the participants thus far, but
> I'm assuming you mean something else?

Well, I just meant to continue in this vein and then follow up with
an explicit checkpoint with all the stake-holders.

For example you could write up a brief wiki walking through how an
existing widely-consumed notification might look under your vision,
say compute.instance.start.end. Then post a link back here as an RFC.

Or, possibly better, maybe submit up a strawman spec proposal to one
of the relevant *-specs repos and invite folks to review in the usual
way?

Anyway, whatever works for you ...

> > If you can figure out how to do that in a concrete way ... then great,
> > I'd be receptive, let's hear some proposals.
> 
> I've made a very limited one, how's it sound? Does it need an
> implementation in order to warrant further discussion? Or would it
> be better to toss it around a bit more? It makes no sense to me to
> formalize something that has no potential legs.

So it sounds like we're converging towards a common understanding,
and while I don't think there's a need for an implementation to warrant
further discussion, a more detailed and concrete (but not necessarily
complete) write-up in say one of the formats suggested above would be
great to keep up the momentum of the discussion.

This feels like something that we should be thinking about with an eye
to the K* cycle - would you agree?

Cheers,
Eoghan