Open Stack

On 11/23/15 at 08:54am, Ryan Rossiter wrote:
>
>
>On 11/23/2015 5:33 AM, John Garbutt wrote:
>>On 20 November 2015 at 09:37, Balázs Gibizer
>><balazs.gibizer at ericsson.com> wrote:
>>><snip>
>>>Minor version change shall not cause any problem for the consumer as the payload is backward compatible between minor versions. So if the consumer does not need the new field then he/she does not need to change anything in his/her parser. As far as I know we had a single major object version in nova so far so this is not a frequent event.  In case of major change I think we can offer version pinning from nova via configuration as a future step.
>>>
>>>The library idea has the problem that it would be python lib and consumers can be in any language. For me lib would be used for compat code but as I mentioned above incompatibility is not that frequent. In the other hand discoverability of notifications are more important for me. For that I can suggest providing notification samples as a first step so the consumer can see in the source tree what notifications are provided by the nova. As a natural next step would be to provide not just samples but schemas for the notifications. I haven't looked it too deep but I feel if we provide json schema for the notifications then the consumer can generate an object model from that schema without too much effort. It might not help directly with backporting the payload but at least automate things around it. The json schema has the benefit that it is language independent too.
>>>
>>So, on re-reading, I think we are missing some of the context from the
>>summit session in this spec:
>>http://specs.openstack.org/openstack/nova-specs/specs/mitaka/approved/versioned-notification-api.html
>>
>>The aim of notifications is to focus more on structured logging, for
>>the operator.
>>
>>It is not trying to provide an end-user async API, such as:
>>https://etherpad.openstack.org/p/liberty-cross-project-user-notifications
>>
>>As Gibi mentions, the idea was to add a version, so you can tell when
>>the content has changed, generally in a backwards compatible way. The
>>other big parts, like properly testing the notifications to ensure
>>they stay compatible.
>>
>>It uses o.vo, with the intent we avoid a major version bump. Should we
>>need one, its likely to end up with a similar transition phase to the
>>current un-versioned->versioned transition phase (effectively the
>>deployer decides which major versions will be emitted at any one
>>time).
>>
>>The interface is a stable JSON structure, with some handy version and
>>identification metadata. It happens to use o.vo to generate that
>>format, and we happen to have python code to read that format, but
>>thats not the aim.
>This is a stupid question but... who's the object versioning designed 
>for? Is it for the consumer, so they know what they're getting? Or is 
>it for Nova, so it has a contract to fill, instead of tossing a big 
>bag of dicts across the wire? Those sound the same, but, suddenly in 
>my head, they aren't.
>
>In the case of the former, if we add a new field, we bump the 
>version, so the consumer knows when it can start looking for new 
>things when it gets that version.
>
>In the case of the latter, the fields defined in version 1.0 is the 
>only guaranteed law until the end of the universe. Every field added 
>after 1.0 needs to be nullable to allow backversioned computes to 
>still work, right? We're still handcuffed on what we can actually 
>truly guarantee the consumer.
>>
>>There is a bit I am conflicted/worried about, and thats when we start
>>including verbatim, DB objects into the notifications. At least you
>>can now quickly detect if that blob is something compatible with your
>>current parsing code. My preference is really to keep the
>>Notifications as a totally separate object tree, but I am sure there
>>are many cases where that ends up being seemingly stupid duplicate
>>work. I am not expressing this well in text form :(
>Are you saying we don't want to be willy-nilly tossing DB objects 
>across the wire? Yeah that was part of the rug-pulling of just having 
>the payload contain an object. We're automatically tossing everything 
>with the object then, whether or not some of that was supposed to be 
>a secret. We could add some sort of property to the field like 
>dont_put_me_on_the_wire=True (or I guess a notification_ready() 
>function that helps an object sanitize itself?) that the 
>notifications will look at to know if it puts that on the 
>wire-serialized dict, but that's adding a lot more complexity and 
>work to a pile that's already growing rapidly.

I don't want to be tossing db objects across the wire.  But I also am 
not convinced that we should be tossing the current objects over the 
wire either.  You make the point that there may be things in the object 
that shouldn't be exposed, and I think object version bumps is another 
thing to watch out for.  So far the only object that has been bumped is 
Instance but in doing so no notifications needed to change.  I think if 
we just put objects into notifications we're coupling the notification 
versions to db or RPC changes unnecessarily.  Some times they'll move 
together but other times, like moving flavor into instance_extra, 
there's no reason to bump notifications.

Note that I'm not against using objects for notifications and 
versioning, but I picture having something like an InstanceNotification 
object which can handle converting an Instance object into a suitable 
notification version/format.

>>
>>Thanks,
>>John
>>
>>>Cheers,
>>>Gibi
>>>>--
>>>>
>>>>Thanks,
>>>>
>>>>Matt Riedemann
>>>
>>>__________________________________________________________________________
>>>OpenStack Development Mailing List (not for usage questions)
>>>Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>__________________________________________________________________________
>>OpenStack Development Mailing List (not for usage questions)
>>Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>-- 
>Thanks,
>
>Ryan Rossiter (rlrossit)
>
>
>__________________________________________________________________________
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev