Open Stack

> On 9/04/2014 Sandy Walsh wrote:
> Yesterday, we had a great conversation with Matt Rutkowski from IBM,
> one
> of the authors of the CADF spec.
> 
> I was having a disconnect on what CADF offers and got it clarified.
> 
> My assumption was CADF was a set of transformation/extraction rules
> for
> taking data from existing data structures and defining them as
> well-known things. For example, CADF needs to know "who sent this
> notification". I thought CADF would give us a means to point at an
> existing data structure and say "that's where you find it".
> 
> But I was wrong. CADF is a full-on schema/data structure of its own. It
> would be a fork-lift replacement for our existing notifications.

This was my "aha" as well, following a similar discussion with Matt and team, but also note that they've articulated an approach for bolt-on changes that would enable CADF content in existing pipelines. (https://wiki.openstack.org/wiki/Ceilometer/blueprints/support-standard-audit-formats)

> However, if your service hasn't really adopted notifications yet (green
> field) or you can handle a fork-lift replacement, CADF is a good option.
> There are a few gotcha's though. If you have required data that is
> outside of the CADF spec, it would need to go in the "attachment"
> section of the notification and that still needs a separate schema to
> define it. Matt's team is very receptive to extending the spec to
> include these special cases though.
Agreed that Matt's team was very willing to extend, but I still wonder about having to migrate appended data from its "pre-approval" location to its "permanent" location, depending on the speed of the CADF standard update.
> 
> Anyway, I've written up all the options (as I see them) [1] with the
> advantages/disadvantages of each approach. It's just a strawman, so
> bend/spindle/mutilate.
Cool...will add comments there.
> 
> Look forward to feedback!
> -S
> 
> 
> [1] https://wiki.openstack.org/wiki/NotificationsAndCADF
> 
> 
> 
> 
> On 9/3/2014 12:30 PM, Sandy Walsh wrote:
> > On 9/3/2014 11:32 AM, Chris Dent wrote:
> >> On Wed, 3 Sep 2014, Sandy Walsh wrote:
> >>
> >>> We're chatting with IBM about CADF and getting down to specifics
> on
> >>> their applicability to notifications. Once I get StackTach.v3 into
> >>> production I'm keen to get started on revisiting the notification
> >>> format and olso.messaging support for notifications.
> >>>
> >>> Perhaps a hangout for those keenly interested in doing something
> about this?
> >> That seems like a good idea. I'd like to be a part of that.
I would , too, and I would suggest that much of the Ceilometer team would 
> >> Unfortunately I won't be at summit but would like to contribute what
> >> I can before and after.
> >>
> >> I took some notes on this a few weeks ago and extracted what
> seemed
> >> to be the two main threads or ideas the were revealed by the
> >> conversation that happened in this thread:
> >>
> >>      * At the micro level have versioned schema for notifications such
> that
> >>        one end can declare "I am sending version X of notification
> >>        foo.bar.Y" and the other end can effectively deal.
> > Yes, that's table-stakes I think. Putting structure around the payload
> > section.
> >
> > Beyond type and version we should be able to attach meta information
> > like public/private visibility and perhaps hints for external mapping
> > (this trait -> that trait in CADF, for example).
> >
> >>      * At the macro level standardize a packaging or envelope of all
> >>        notifications so that they can be consumed by very similar code.
> >>        That is: constrain the notifications in some way so we can also
> >>        constrain the consumer code.
> > That's the intention of what we have now. The top level traits are
> > standard, the payload is open. We really only require: message_id,
> > timestamp and event_type. For auditing we need to cover Who, What,
> When,
> > Where, Why, OnWhat, OnWhere, FromWhere.
> >

To whit, I think we've made good progress in this by defining the "what is the minimum content" for PaaS service notifications and gotten agreement around  https://review.openstack.org/#/c/113396/11/doc/source/format.rst
for the Juno release. It's been driven by many of these same questions but is fairly narrow in scope; it defines a minimum set of content, but doesn't tackle the question of structure (beyond trait typing). The timing seems right to dig deeper.

> >>      These ideas serve two different purposes: One is to ensure that
> >>      existing notification use cases are satisfied with robustness and
> >>      provide a contract between two endpoints. The other is to allow a
> >>      fecund notification environment that allows and enables many
> >>      participants.
> > Good goals. When Producer and Consumer know what to expect,
> things are
> > good ... "I know to find the Instance ID <here>". When the consumer
> > wants to deal with a notification as a generic object, things get tricky
> > ("find the instance ID in the payload", "What is the image type?", "Is
> > this an error notification?")
> >
> > Basically, how do we define the principle artifacts for each service and
> > grant the consumer easy/consistent access to them? (like the 7-W's
> above)
> >
> > I'd really like to find a way to solve that problem.
> >
> >> Is that a good summary? What did I leave out or get wrong?
> >>
> > Great start! Let's keep it simple and do-able.
> >
> > We should also review the oslo.messaging notification api ... I've got
> > some concerns we've lost our way there.
> >
> > -S
> >
> >
> > _______________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev