Open Stack

On 8/28/15, 10:07 AM, "gord chung" <gord at live.ca> wrote:

>
>
>On 28/08/15 12:18 PM, Roman Dobosz wrote:
>> So imagine we have new versions of the schema for the events, alarms or
>> samples in ceilometer introduced in Mitaka release while you have all
>> your ceilo services on Liberty release. To upgrade ceilometer you'll
>> have to stop all services to avoid data corruption. With
>> versionedobjects you can do this one by one without disrupting
>> telemetry jobs.
>are versions checked for every single message? has anyone considered the
>overhead to validating each message? since ceilometer is queue based, we
>could technically just publish to a new queue when schema changes... and
>the consuming services will listen to the queue it knows of.
>
>ie. our notification service changes schema so it will now publish to a
>v2 queue, the existing collector service consumes the v1 queue until
>done at which point you can upgrade it and it will listen to v2 queue.
>
>this way there is no need to validate/convert anything and you can still
>take services down one at a time. this support doesn't exist currently
>(i just randomly thought of it) but assuming there's no flaw in my idea
>(which there may be) isn't this more efficient?

If high performance is a concern for ceilometer (and it should) then maybe
there might be better options than JSON?
JSON is great for many applications but can be inappropriate for other
demanding applications.
There are other popular open source encoding options that yield much more
compact wire payload, more efficient encoding/decoding and handle
versioning to a reasonable extent.

Queue based versioning might be less runtime overhead per message but at
the expense of a potentially complex queue version management (which can
become tricky if you have more than 2 versions).
I think Neutron was considering to use versioned queues as well for its
rolling upgrade (along with versioned objects) and I already pointed out
that managing the queues could be tricky.

In general, trying to provide a versioning framework that allows to do
arbitrary changes between versions is quite difficult (and often bound to
fail).




>>
>> The other thing, maybe not so obvious, is to put versionedobject layer
>> between application and the MongoDB driver, so that all of the schema
>> changes will be automatically handled on ovo, and also serialization
>> might also be done on such layer.
>
>i don't quite understand this, is this a mongodb specific solution?
>admittedly the schemaless design of mongo i can imagine causing issues
>but currently we're trying to avoid wasting resources on existing
>mongodb solution as we attempt to move to new api. if it's just a
>generic db solution, i'd be interested to apply it to future designs.

I'd suggest to do some benchmarking to make sure the cost of
serializing/deserializing with versioned objects is not prohibitive.

  Alec


>
>cheers,
>
>-- 
>gord
>
>
>__________________________________________________________________________
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev