Open Stack

>________________________________________
>From: Eoghan Glynn [eglynn at redhat.com] Thursday, November 20, 2014 5:34 PM
>
>Some questions/observations inline.
>
>> Hey y'all,
>>
>> To avoid cross-posting, please inform your -infra / -operations buddies about this post.
>>
>> We've just started thinking about where notification schema files should live and how they should be deployed. Kind of a tricky problem.  We could really use your input on this problem ...
>>
>> The assumptions:
>> 1. Schema files will be text files. They'll live in their own git repo (stackforge for now, ideally oslo eventually).
>> 2. Unit tests will need access to these files for local dev
>> 3. Gating tests will need access to these files for integration tests
>> 4. Many different services are going to want to access these files during staging and production.
>> 5. There are going to be many different versions of these files. There are going to be a lot of schema updates.
>>
>> Some problems / options:
>> a. Unlike Python, there is no simple pip install for text files. No version control per se. Basically whatever we pull from the repo. The problem with a git clone is we need to tweak config files to point to a directory and that's a pain for gating tests and CD. Could we assume a symlink to some well-known location?
>>     a': I suppose we could make a python installer for them, but that's a pain for other language consumers.

>Would it be unfair to push that burden onto the writers of clients
>in other languages?
>
>i.e. OpenStack, being largely python-centric, would take responsibility
>for both:
>
>  1. Maintaining the text versions of the schema in-tree (e.g. as json)
>
>and:
>
>  2. Producing a python-specific installer based on #1
>
>whereas, the first Java-based consumer of these schema would take
>#1 and package it up in their native format, i.e. as a jar or
>OSGi bundle.

Certainly an option. My gut says it will lead to abandoned/fragmented efforts.
If I was a ruby developer, would I want to take on the burden of maintaining yet another package? 
I think we need to treat this data as a form of API and there it's our responsibility to make easily consumable. 

(I'm not hard-line on this, again, just my gut feeling)

>> b. In production, each openstack service could expose the schema files via their REST API, but that doesn't help gating tests or unit tests. Also, this means every service will need to support exposing schema files. Big coordination problem.

>I kind of liked this schemaURL endpoint idea when it was first
>mooted at summit.
>
>The attraction for me was that it would allow the consumer of the
>notifications always have access to the actual version of schema
>currently used on the emitter side, independent of the (possibly
>out-of-date) version of the schema that the consumer has itself
>installed locally via a static dependency.
>
>However IIRC there were also concerns expressed about the churn
>during some future rolling upgrades - i.e. if some instances of
>the nova-api schemaURL endpoint are still serving out the old
>schema, after others in the same deployment have already been
>updated to emit the new notification version.

Yeah, I like this idea too. In the production / staging phase this seems like the best route. The local dev / testing situation seems to be the real tough nut to crack. 

WRT rolling upgrades we have to ensure we update the service catalog first, the rest should be fine. 

>> c. In production, We could add an endpoint to the Keystone Service Catalog to each schema file. This could come from a separate metadata-like service. Again, yet-another-service to deploy and make highly available.

>Also to {puppetize|chef|ansible|...}-ize.
>
>Yeah, agreed, we probably don't want to do down that road.

Which is kinda unfortunate since it's the lowest impact on other projects. 

>> d. Should we make separate distro packages? Install to a well known location all the time? This would work for local dev and integration testing and we could fall back on B and C for production distribution. Of course, this will likely require people to add a new distro repo. Is that a concern?

>Quick clarification ... when you say "distro packages", do you mean
>Linux-distro-specific package formats such as .rpm or .deb?

Yep. 

>Cheers,
>Eoghan

Thanks for the feedback!