Open Stack

i should start by saying i re-read my subject line and it arguably comes 
off aggressive -- i should probably have dropped 'explain' :)

On 28/08/15 01:47 PM, Alec Hothan (ahothan) wrote:
>
> On 8/28/15, 10:07 AM, "gord chung" <gord at live.ca> wrote:
>
>>
>> On 28/08/15 12:18 PM, Roman Dobosz wrote:
>>> So imagine we have new versions of the schema for the events, alarms or
>>> samples in ceilometer introduced in Mitaka release while you have all
>>> your ceilo services on Liberty release. To upgrade ceilometer you'll
>>> have to stop all services to avoid data corruption. With
>>> versionedobjects you can do this one by one without disrupting
>>> telemetry jobs.
>> are versions checked for every single message? has anyone considered the
>> overhead to validating each message? since ceilometer is queue based, we
>> could technically just publish to a new queue when schema changes... and
>> the consuming services will listen to the queue it knows of.
>>
>> ie. our notification service changes schema so it will now publish to a
>> v2 queue, the existing collector service consumes the v1 queue until
>> done at which point you can upgrade it and it will listen to v2 queue.
>>
>> this way there is no need to validate/convert anything and you can still
>> take services down one at a time. this support doesn't exist currently
>> (i just randomly thought of it) but assuming there's no flaw in my idea
>> (which there may be) isn't this more efficient?
> If high performance is a concern for ceilometer (and it should) then maybe
> there might be better options than JSON?
> JSON is great for many applications but can be inappropriate for other
> demanding applications.
> There are other popular open source encoding options that yield much more
> compact wire payload, more efficient encoding/decoding and handle
> versioning to a reasonable extent.

i should clarify. we let oslo.messaging serialise our dictionary how it 
does... i believe it's JSON. i'd be interested to switch it to something 
more efficient. maybe it's time we revive the msgpacks patch[1] or are 
there better alternatives? (hoping i didn't just unleash a storm of 
'this is better' replies)

>
> Queue based versioning might be less runtime overhead per message but at
> the expense of a potentially complex queue version management (which can
> become tricky if you have more than 2 versions).
> I think Neutron was considering to use versioned queues as well for its
> rolling upgrade (along with versioned objects) and I already pointed out
> that managing the queues could be tricky.
>
> In general, trying to provide a versioning framework that allows to do
> arbitrary changes between versions is quite difficult (and often bound to
> fail).
>
yeah, so that's what a lot of the devs are debating about right now. 
performance is our key driver so if we do something we think/know will 
negatively impact performance, it better bring a whole lot more of 
something else. if queue based versioning offers comparable 
functionalities, i'd personally be more interested to explore that route 
first. is there a thread/patch/log that we could read to see what 
Neutron discovered when they looked into it?

[1] https://review.openstack.org/#/c/151301/

-- 
gord