Open Stack

On 11/27/2012 06:39 AM, Adam Young wrote:
> On 11/26/2012 05:33 PM, Mark McLoughlin wrote:
>> On Thu, 2012-11-22 at 23:21 -0800, Monty Taylor wrote:
>>> On 11/22/2012 09:44 AM, Mark McLoughlin wrote:
>>>> Well, to add two more questions:
>>>>
>>>>     * When do we commit to compatibility around a new API in Oslo? When
>>>>       it gets committed, when it gets released or when we release an
>>>>       officially "API frozen" version?
>>> I vote for when it gets released.
>>>
>>>>     * How do other OpenStack projects consume Oslo libraries during
>>>>       development? e.g. can you put a git URL into pip-requires, or
>>>> must
>>>>       you have a pypi release version there?
>>>>
>>>> You could say e.g. new APIs are committed to when they're released to
>>>> pypi and OpenStack projects can only consume new APIs once they've been
>>>> released to pypi.
>>> Yes please. We have done cross-project git-url consumption before. It's
>>> a nightmare (largely because although the python toolchain tempts us
>>> into thinking this will work, it doesn't actually fully work.
>>>
>>>> But that could result us doing library releases too often, integration
>>>> with other projects being too slow or us only realizing we've made
>>>> drastic API design mistakes after committing to the new API.
>>> Honestly, I think you have a slightly easier time of it than the client
>>> lib folks in terms of API commitment - since you know who your consumers
>>> are, and you can actually check around to see if anyone is using an old
>>> API. It's not too terrible - how about we try it that way for a bit and
>>> see if there are actual logistical problems?
>> Leaving aside mechanics for a second, the more I think about this the
>> more I'm coming to a pretty firm view ...
>>
>> I want the 6 month release cycle model for Oslo. I want the project to
>> go through a bunch of hacking for the first part of that cycle and lock
>> things down towards the end of the release. I want Oslo to have a stable
>> branch where people can get fixes without the risk of new features. I
>> want milestone release checkpoints to focus the mind and not have one
>> long period of hacking before a release. I want Oslo's cycle to be in
>> sync with the projects which are using Oslo's APIs. I want people to
>> intuitively understand what Oslo's cycle is rather than it having a
>> bunch of special rules. And I want Thierry's help coordinating all
>> this :-)
> Keystone will help, too.  Keystone is probably the closest project to
> Oslo, in that it holds many pieces that other projects are going to want.
>
>>
>> Basically, my instinct is to do everything just like any of the other
>> core projects except in those cases where Oslo really is a special case.
>>
>> The main difference is other core projects are consuming APIs provided
>> by Oslo.
>>
>> I think it makes sense to say other projects can only consume new Oslo
>> APIs if they've actually been released.
>>
>> I wouldn't like to see Oslo development releases go on pypi. To me, that
>> would give those releases an inaccurate appearance of stability. I would
>> like to see stable branch releases going to pypi.
>
> That sounds right.  We definitely need shakedown time.

I'd like to see oslo libraries treated like libraries.

That's not necessarily at odds with the core of what you said, I think 
it's important to think about what it means to have  6-month lead time 
between landing a feature in oslo and it being able to be used in any of 
the projects. So I agree that other projects should only consume new 
oslo apis if they've been released, and I agree about not publishing dev 
versions to PyPI - I would counsel against setting them on the 
hard-and-fast rule of the 6-month core project release cycle. 
(obviously, you can always NOT release something if it's not ready)

Although I do think we _can_ solve the logistics if we decide to go this 
route, having one of our core projects actually require dev releases of 
libraries during their dev cycle gets us right back to the place we were 
a year ago with client libs and that we worked very hard to get out of. 
So please reconsider anything which leads you down the path of wanting 
to have one of our projects require a git-version of another of our 
projects.

>> During development, I think we should have some wiggle room to fix up
>> bad API design errors even after we've done a development release with
>> those errors. If we can notice the error, add a better API, deprecate
>> the broken API, update all the core projects and remove the broken API
>> before final release, I think that's workable.
>
> Agreed.  What I expect to see is something like this:
>
> Project X makes a feature work.
> That feature is now needed by project Y
> Feature moves over to Oslo for consumtion by project Y
> Additional input from Project Y developers lead to changes of the feature
>
>
> An  example:Swift has worked on abstracting the Map interface.  It can
> be memcached, or a pure in memory one.  They want that work in Keystone
> auth_token.  The right route is through Keystone, but we need to make
> sure that it comes in clean.
>
>
>
>>
>> Can we figure out the mechanics to make this all possible? e.g. can we
>> push tarball releases somewhere during development other than PyPi and
>> just reference those tarball URLs directly in pip-requires? At release
>> time, we'd push Oslo releases to PyPi and update the core projects to
>> use those as their dependencies rather than URLs.
>
> Is this something that we can do via git?  I'm thinking along the lines
> of how Keystone checks out python-keystone client in the keystone/vendor
> directory when it is running.

Let's come back to the how after we decide the what. ... there are a 
bunch of interesting requirements this is going to have. It's quite 
possible something like keystone's vendor thing can be helpful, but I'd 
love to collect the entire problem space and circle up with people and 
make a comprehensive solution.