Open Stack

> From: Ryan Rossiter [mailto:rlrossit at linux.vnet.ibm.com]
> Sent: November 23, 2015 15:54
> On 11/23/2015 5:33 AM, John Garbutt wrote:
> > On 20 November 2015 at 09:37, Balázs Gibizer
> > <balazs.gibizer at ericsson.com> wrote:
> >> <snip>
> >> Minor version change shall not cause any problem for the consumer as
> the payload is backward compatible between minor versions. So if the
> consumer does not need the new field then he/she does not need to
> change anything in his/her parser. As far as I know we had a single major
> object version in nova so far so this is not a frequent event.  In case of major
> change I think we can offer version pinning from nova via configuration as a
> future step.
> >>
> >> The library idea has the problem that it would be python lib and
> consumers can be in any language. For me lib would be used for compat code
> but as I mentioned above incompatibility is not that frequent. In the other
> hand discoverability of notifications are more important for me. For that I can
> suggest providing notification samples as a first step so the consumer can see
> in the source tree what notifications are provided by the nova. As a natural
> next step would be to provide not just samples but schemas for the
> notifications. I haven't looked it too deep but I feel if we provide json schema
> for the notifications then the consumer can generate an object model from
> that schema without too much effort. It might not help directly with
> backporting the payload but at least automate things around it. The json
> schema has the benefit that it is language independent too.
> >>
> > So, on re-reading, I think we are missing some of the context from the
> > summit session in this spec:
> > http://specs.openstack.org/openstack/nova-
> specs/specs/mitaka/approved/
> > versioned-notification-api.html
> >
> > The aim of notifications is to focus more on structured logging, for
> > the operator.
> >
> > It is not trying to provide an end-user async API, such as:
> > https://etherpad.openstack.org/p/liberty-cross-project-user-notificati
> > ons
> >
> > As Gibi mentions, the idea was to add a version, so you can tell when
> > the content has changed, generally in a backwards compatible way. The
> > other big parts, like properly testing the notifications to ensure
> > they stay compatible.
> >
> > It uses o.vo, with the intent we avoid a major version bump. Should we
> > need one, its likely to end up with a similar transition phase to the
> > current un-versioned->versioned transition phase (effectively the
> > deployer decides which major versions will be emitted at any one
> > time).
> >
> > The interface is a stable JSON structure, with some handy version and
> > identification metadata. It happens to use o.vo to generate that
> > format, and we happen to have python code to read that format, but
> > thats not the aim.
> This is a stupid question but... who's the object versioning designed for? Is it
> for the consumer, so they know what they're getting? Or is it for Nova, so it
> has a contract to fill, instead of tossing a big bag of dicts across the wire?
> Those sound the same, but, suddenly in my head, they aren't.
> 
> In the case of the former, if we add a new field, we bump the version, so the
> consumer knows when it can start looking for new things when it gets that
> version.
Currently if we add a field we bump the minor version, so if the consumer want to look into the new content he or she can detect that there is new content.

> 
> In the case of the latter, the fields defined in version 1.0 is the only
> guaranteed law until the end of the universe. Every field added after
> 1.0 needs to be nullable to allow backversioned computes to still work, right?
> We're still handcuffed on what we can actually truly guarantee the
> consumer.
I don't fully understand this comment. If the consumer prepared to consume a payload with two fields as version 1.0, when we add a new field and bump the version to 1.1 the consumer's old parser can still work without change (if it is written well) and it will found all the data the 1.0 view needs.  If we decides to remove a field from the payload and therefore bump the version to 2.0 then the consumer's parser will fail so we have to avoid that. In this case it is nova responsibility to offer a solution and I agree with John that a transition period with keeping the old 1.1 version will work. 

> >
> > There is a bit I am conflicted/worried about, and thats when we start
> > including verbatim, DB objects into the notifications. At least you
> > can now quickly detect if that blob is something compatible with your
> > current parsing code. My preference is really to keep the
> > Notifications as a totally separate object tree, but I am sure there
> > are many cases where that ends up being seemingly stupid duplicate
> > work. I am not expressing this well in text form :(
> Are you saying we don't want to be willy-nilly tossing DB objects across the
> wire? Yeah that was part of the rug-pulling of just having the payload contain
> an object. We're automatically tossing everything with the object then,
> whether or not some of that was supposed to be a secret. We could add
> some sort of property to the field like dont_put_me_on_the_wire=True (or I
> guess a notification_ready() function that helps an object sanitize itself?) that
> the notifications will look at to know if it puts that on the wire-serialized dict,
> but that's adding a lot more complexity and work to a pile that's already
> growing rapidly.
We are not necessarily pushing everything along with the object, we can filter out fields from the existing objects when we use them as payload notifications.  This is what is currently implemented with the _remove_filtered_fields of the NotificationPayloadBase object. This does not seems like a big work to implement.
The other way would be to duplicate model. Keep the existing internal model and create a new, separate model for notifications and build a translation mechanism between the two. This seems a more complex solution to me but it can be done. Do you suggest to go this direction?

> >
> > Thanks,
> > John
> >
> >> Cheers,
> >> Gibi
> >>> --
> >>>
> >>> Thanks,
> >>>
> >>> Matt Riedemann
> >>
> >>
> __________________________________________________________
> ___________
> >> _____ OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe:
> >> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> __________________________________________________________
> ____________
> > ____ OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> > OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> --
> Thanks,
> 
> Ryan Rossiter (rlrossit)
> 
> 
> __________________________________________________________
> ________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-
> request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev