Open Stack

Excerpts from Sean Dague's message of 2015-06-17 14:07:35 -0400:
> On 06/17/2015 01:29 PM, Clint Byrum wrote:
> > Excerpts from Sean Dague's message of 2015-06-16 10:16:34 -0700:
> >> On 06/16/2015 12:49 PM, Clint Byrum wrote:
> >>> Excerpts from Sean Dague's message of 2015-06-16 06:22:23 -0700:
> >>>> FYI,
> >>>>
> >>>> One of the things that came out of the summit for Devstack plans going
> >>>> forward is to trim it back to something more opinionated and remove a
> >>>> bunch of low use optionality in the process.
> >>>>
> >>>> One of those branches to be trimmed is all the support for things beyond
> >>>> RabbitMQ in the rpc layer. RabbitMQ is what's used by 95%+ of our
> >>>> community, that's what the development environment should focus on.
> >>>>
> >>>> The patch to remove all of this is here -
> >>>> https://review.openstack.org/#/c/192154/. Expect this to merge by the
> >>>> end of the month. If people are interested in non RabbitMQ external
> >>>> plugins, now is the time to start writing them. The oslo.messaging team
> >>>> already moved their functional test installation for alternative
> >>>> platforms off of devstack, so this should impact a very small number of
> >>>> people.
> >>>>
> >>>
> >>> The recent spec we added to define a policy for oslo.messaging drivers is
> >>> intended as a way to encourage that 5% who feels a different messaging
> >>> layer is critical to participate upstream by adding devstack-gate jobs
> >>> and committing developers to keep them stable. This change basically
> >>> slams the door in their face and says "good luck, we don't actually care
> >>> about accomodating you." This will drive them more into the shadows,
> >>> and push their forks even further away from the core of the project. If
> >>> that's your intention, then we need to have a longer conversation where
> >>> you explain to me why you feel that's a good thing.
> >>
> >> I believe it is not the responsibility of the devstack team to support
> >> every possible backend one could imagine and carry that technical debt
> >> in tree, confusing new users in the process that any of these things
> >> might actually work. I believe that if you feel that your spec assumed
> >> that was going to be the case, you made a large incorrect externalities
> >> assumption.
> >>
> > 
> > I agree with you, and support your desire to move things into plugins.
> > 
> > However, your timing is problematic and the lack of coordination with
> > the ongoing effort to deprecate untested messaging drivers gracefully
> > is really frustrating. We've been asking (on this list) zmq interested
> > parties to add devstack-gate jobs and identify themselves as contacts
> > to support these drivers. Meanwhile this change and the wording around
> > it suggest that they're not welcome in devstack.
> 
> So there has clearly been some disconnect here. This patch was
> originally going to come later in the cycle, but some back and forth on
> proton fixes with Flavio made me realize we really needed to get this
> direction out in front of more people (which is why it wasn't just a
> patch, it was also an email heads up). So there wasn't surprise when it
> was merged.
> 
> We built the external plugin mechanism in devstack to make it very easy
> to extend out of tree, and make it easy to let people consume your out
> of tree stuff. It's the only way that devstack works in the big tent
> world, because there just is too much stuff for the team to support.

Every change like this makes it harder for newcomers to participate.
Frankly, it makes it harder for everyone because it means there are
more moving parts, but in this specific case many of the people
involved in these messaging drivers are relatively new, so I point
that out. The already difficult task of setting up sufficient
functional tests has now turned into "figure out devstack", too.
The long-term Oslo team members can't do all of this work, any more
than the devstack team can, but things were at least working in
what we thought was a stable way so we could try to provide guidance.

> 
> >>> Also, I take issue with the value assigned to dropping it. If that 95%
> >>> is calculated as orgs_running_on_rabbit/orgs then it's telling a really
> >>> lop-sided story. I'd rather see compute_nodes_on_rabbit/compute_nodes.
> >>>
> >>> I'd like to propose that we leave all of this in tree to match what is
> >>> in oslo.messaging. I think devstack should follow oslo.messaging and
> >>> deprecate the ones that oslo.messaging deprecates. Otherwise I feel like
> >>> we're Vizzini cutting the rope just as The Dread Pirate 0mq is about to
> >>> climb the last 10 meters to the top of the cliffs of insanity and battle
> >>> RabbitMQ left handed. I know, "inconceivable" right?
> >>
> >> We have an external plugin mechanism for devstack. That's a viable
> >> option here. People will have to own and do that work, instead of
> >> expecting the small devstack team to do that for them. I believe I left
> >> enough of a hook in place that it's possible.
> >>
> > 
> > So lets do some communication, and ask for the qpid and zmq people to
> > step up, and help them move their code into an external plugin, and add
> > documentation to help their users find it. The burden should shift, but
> > it still rests with devstack until it _does_ shift.
> 
> We still need to set a clock, because in the past when we haven't, the
> burden never shifts.

In the past when we've made big changes like this, we've treated
them as deprecations with long lead times. So, how about the beginning
of the N cycle (or end of M)? That gives the teams time to ramp up
and understand what's going on within Oslo, before having to figure
out the whole world in order to meet the publicly discussed testing
requirements they already knew about.

> 
> >> That would also let them control the code relevant to their plugin,
> >> because there is no way that devstack was going to gate against other
> >> backends here, so we'd end up breaking them pretty often, and it taking
> >> a while to fix them in tree.
> > 
> > I love that idea. That is not what the change does though. It deletes
> > with nary a word about what users of this code should do until new
> > external plugins appear.
> 
> Sure, lets get folks engaged now. I'm happy to help people debug this
> code to get it working. The burden of the effort does need to be on the
> folks with the feature they want to easily enable in devstack, but we've
> spent a lot of time getting this stuff working for a lot of different
> use cases. It should only be a couple of days of poking, even for
> someone pretty new at this.

Does the existing code in devstack not work? Is retaining it
preventing other work the devstack team wants to undertake? Does
this have to be done *right now*? Because, as Clint pointed out, the
timing is really bad for the people your change affects.

> If we have a "pretty close" attempt by a team, I'll happily work out any
> last mile problems. That actually worked out really great with the
> modular grenade / heat effort, as all the last mile bits got sorted in 2
> hrs time because there was mostly functional bits on both sides of the
> fence.

Is there documentation for the new plugin system? Are there examples
available?

Doug

> 
> But we do need people thinking *now* about writing external plugins, and
> make that part of the workflow.
> 
>     -Sean
>