Open Stack

On 17/06/15 15:08 -0400, Doug Hellmann wrote:
>Excerpts from Sean Dague's message of 2015-06-17 14:07:35 -0400:
>> On 06/17/2015 01:29 PM, Clint Byrum wrote:
>> > Excerpts from Sean Dague's message of 2015-06-16 10:16:34 -0700:
>> >> On 06/16/2015 12:49 PM, Clint Byrum wrote:
>> >>> Excerpts from Sean Dague's message of 2015-06-16 06:22:23 -0700:
>> >>>> FYI,
>> >>>>
>> >>>> One of the things that came out of the summit for Devstack plans going
>> >>>> forward is to trim it back to something more opinionated and remove a
>> >>>> bunch of low use optionality in the process.
>> >>>>
>> >>>> One of those branches to be trimmed is all the support for things beyond
>> >>>> RabbitMQ in the rpc layer. RabbitMQ is what's used by 95%+ of our
>> >>>> community, that's what the development environment should focus on.
>> >>>>
>> >>>> The patch to remove all of this is here -
>> >>>> https://review.openstack.org/#/c/192154/. Expect this to merge by the
>> >>>> end of the month. If people are interested in non RabbitMQ external
>> >>>> plugins, now is the time to start writing them. The oslo.messaging team
>> >>>> already moved their functional test installation for alternative
>> >>>> platforms off of devstack, so this should impact a very small number of
>> >>>> people.
>> >>>>
>> >>>
>> >>> The recent spec we added to define a policy for oslo.messaging drivers is
>> >>> intended as a way to encourage that 5% who feels a different messaging
>> >>> layer is critical to participate upstream by adding devstack-gate jobs
>> >>> and committing developers to keep them stable. This change basically
>> >>> slams the door in their face and says "good luck, we don't actually care
>> >>> about accomodating you." This will drive them more into the shadows,
>> >>> and push their forks even further away from the core of the project. If
>> >>> that's your intention, then we need to have a longer conversation where
>> >>> you explain to me why you feel that's a good thing.
>> >>
>> >> I believe it is not the responsibility of the devstack team to support
>> >> every possible backend one could imagine and carry that technical debt
>> >> in tree, confusing new users in the process that any of these things
>> >> might actually work. I believe that if you feel that your spec assumed
>> >> that was going to be the case, you made a large incorrect externalities
>> >> assumption.
>> >>
>> >
>> > I agree with you, and support your desire to move things into plugins.
>> >
>> > However, your timing is problematic and the lack of coordination with
>> > the ongoing effort to deprecate untested messaging drivers gracefully
>> > is really frustrating. We've been asking (on this list) zmq interested
>> > parties to add devstack-gate jobs and identify themselves as contacts
>> > to support these drivers. Meanwhile this change and the wording around
>> > it suggest that they're not welcome in devstack.
>>
>> So there has clearly been some disconnect here. This patch was
>> originally going to come later in the cycle, but some back and forth on
>> proton fixes with Flavio made me realize we really needed to get this
>> direction out in front of more people (which is why it wasn't just a
>> patch, it was also an email heads up). So there wasn't surprise when it
>> was merged.
>>
>> We built the external plugin mechanism in devstack to make it very easy
>> to extend out of tree, and make it easy to let people consume your out
>> of tree stuff. It's the only way that devstack works in the big tent
>> world, because there just is too much stuff for the team to support.
>
>Every change like this makes it harder for newcomers to participate.
>Frankly, it makes it harder for everyone because it means there are
>more moving parts, but in this specific case many of the people
>involved in these messaging drivers are relatively new, so I point
>that out. The already difficult task of setting up sufficient
>functional tests has now turned into "figure out devstack", too.
>The long-term Oslo team members can't do all of this work, any more
>than the devstack team can, but things were at least working in
>what we thought was a stable way so we could try to provide guidance.

This is definitely an unfortunate side-effect of this change and it'll
increase the burden we, oslo.messaging maintainers, will have to
carry. Oslo messaging have moved away from using devstack for the
functional tests, it doesn't even use plugins. However, this won't be
enough to provide proper testing, as we're also requesting o.m drivers
maintainers to have and integrated gate running (which will obviously
require having a devstack plugin for o.m).

In the long run, I think the split would benefit oslo.messaging as
it'll give us more control on what's going on in devstack land for
each driver. I'm personally tired of having patches like this[0]
blocked because devstack is missing some packages.

[0] https://review.openstack.org/#/c/192417/

I know none of us is arguing against the goal but the method. I just
wanted to point something in favor of the change and also say that I
agree we should have a proper deprecation path (keep reading).

>> >>> Also, I take issue with the value assigned to dropping it. If that 95%
>> >>> is calculated as orgs_running_on_rabbit/orgs then it's telling a really
>> >>> lop-sided story. I'd rather see compute_nodes_on_rabbit/compute_nodes.
>> >>>
>> >>> I'd like to propose that we leave all of this in tree to match what is
>> >>> in oslo.messaging. I think devstack should follow oslo.messaging and
>> >>> deprecate the ones that oslo.messaging deprecates. Otherwise I feel like
>> >>> we're Vizzini cutting the rope just as The Dread Pirate 0mq is about to
>> >>> climb the last 10 meters to the top of the cliffs of insanity and battle
>> >>> RabbitMQ left handed. I know, "inconceivable" right?
>> >>
>> >> We have an external plugin mechanism for devstack. That's a viable
>> >> option here. People will have to own and do that work, instead of
>> >> expecting the small devstack team to do that for them. I believe I left
>> >> enough of a hook in place that it's possible.
>> >>
>> >
>> > So lets do some communication, and ask for the qpid and zmq people to
>> > step up, and help them move their code into an external plugin, and add
>> > documentation to help their users find it. The burden should shift, but
>> > it still rests with devstack until it _does_ shift.
>>
>> We still need to set a clock, because in the past when we haven't, the
>> burden never shifts.
>
>In the past when we've made big changes like this, we've treated
>them as deprecations with long lead times. So, how about the beginning
>of the N cycle (or end of M)? That gives the teams time to ramp up
>and understand what's going on within Oslo, before having to figure
>out the whole world in order to meet the publicly discussed testing
>requirements they already knew about.

Agreed, lets have a better timeline for this. Being completely honest,
after talking with Sean and seeing the patch up, I was ok with it and
I didn't consider that a deprecation path for this change should be
provided but it really is necessary. We need to provide drivers
maintainers enough time to create these plugins but we can't ask them
to do that while they are also trying to meet the requirements we just
set up for inclusion in oslo.messaging.

M-2 sounds like a good milestone to me.

[snip]

Cheers,
Flavio
-- 
@flaper87
Flavio Percoco
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150618/e5286b3a/attachment.pgp>