Open Stack

On Tue, Aug 09, 2016 at 09:16:02PM -0700, John Griffith wrote:
> On Tue, Aug 9, 2016 at 7:21 PM, Matthew Treinish <mtreinish at kortar.org>
> wrote:
> 
> > On Tue, Aug 09, 2016 at 05:28:52PM -0700, John Griffith wrote:
> > > On Tue, Aug 9, 2016 at 4:53 PM, Sean McGinnis <sean.mcginnis at gmx.com>
> > wrote:
> > >
> > > > .
> > > > >
> > > > > Mike, you must have left the midcycle by the time this topic came
> > > > > up. On the issue of out-of-tree drivers, I specifically offered this
> > > > > proposal (a community managed mechanism for distributing driver
> > > > > bugfix backports) as an compromise alternative to try to address the
> > > > > needs of both camps. Everyone who was in the room at the time (plus
> > > > > DuncanT who wasn't) agreed that if we had that (a way to deal with
> > > > > backports) that they wouldn't want drivers out of the tree anymore.
> > > > >
> > > > > Your point of view wasn't represented so go ahead and explain why,
> > > > > if we did have a reasonable way for bugfixes to get backported to
> > > > > the releases customers actually run (leaving that mechanism
> > > > > unspecified for the time being), that you would still want the
> > > > > drivers out of the tree.
> > > > >
> > > > > -Ben Swartzlander
> > > >
> > > > The conversation about this started around the 30 minute point here if
> > > > anyone is interested in more of the background discussion on this:
> > > >
> > > > https://www.youtube.com/watch?v=g3MEDFp08t4
> > > >
> > > > ____________________________________________________________
> > ______________
> > > > OpenStack Development Mailing List (not for usage questions)
> > > > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:
> > unsubscribe
> > > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > > >
> > >
> > > ​I don't think anybody is whining at all here, we had a fairly productive
> > > discussion at the mid-cycle surrounding this topic and I do think there
> > are
> > > some valid advantages to this approach regardless of the QA question.
> > Note
> > > that it's been pointed out we weren't talking about or considering
> > > advertising this *special* branch as tested by the standard means or gate
> > > CI etc.
> > >
> > > We did discuss this though mostly in the context of helping the package
> > > maintainers and distributions.  The fact is that many of us currently
> > offer
> > > backports of fixes in our own various github accounts.  That's fine and
> > it
> > > works well for many.  The problem we were trying to address however is
> > that
> > > this practice is rather problematic for the distros.  For example RHEL,
> > > Helion or Mirantis are most certainly not going to run around cherry
> > > picking change sets from random github repos scattered around.
> > >
> > > The context of the discussion was that by having a long lived *driver*
> > > (emphasis on driver) branch there would be a single location and an
> > *easy*
> > > method of contact and communication regarding fixes to drivers that may
> > be
> > > available for stable branches that are no longer supported.  This puts
> > the
> > > burden of QA/Testing mostly on the vendors and distros, which I think is
> > > fine.  They can either choose to work with the Vendor and verify the
> > > versions for backport on a regular basis, or they can choose to ignore
> > them
> > > and NOT provide them to their customers.
> > >
> > > I don't think this is an awful idea, and it's very far from the "drivers
> > > out of tree" discussion.  The feedback from the distro maintainers during
> > > the week was that they would gladly welcome a model where they could pull
> > > updates from a single driver branch on a regular basis or as needed for
> > > customers that are on *unsupported* releases and for whom a fix exists.
> > > Note that support cycles are not the same for the distros as they are of
> > > the upstream community.  This is in no way proposing a change to the
> > > existing support time frames or processes we have now, and in that way it
> > > differs significantly from proposals and discussions we've had in the
> > past.
> > >
> > > The basic idea here was to eliminate the proliferation of custom backport
> > > patches scattered all over the web, and to ease the burden for distros
> > and
> > > vendors in supporting their customers.  I think there may be some
> > concepts
> > > to iron out and I certainly understand some of the comments regarding
> > being
> > > disingenuous regarding what we're advertising.  I think that's a
> > > misunderstanding of the intent however, the proposal is not to extend the
> > > support life of stable from an upstream or community perspective but
> > > instead the proposal is geared at consolidation and tracking of drivers.
> >
> > I fully understood the proposal but I still think you're optimizing for the
> > wrong thing.
> 
> ​Ok, that's fair. It seemed like there might be some confusion with some of
> the comments that were made.
>  ​
> 
> 
> > We have a community process for doing backports and maintaining
> > released versions of OpenStack code. The fundamental problem here is
> > actually
> > that the parties you've identified aren't actively involved in stable
> > branch
> > maintenance.
> 
> ​Yes, I know that I've had this conversation with you as well as a few
> others on the infra team over the past year or so.
> ​
> 
> 
> > The stable maint team and policy was primarily created as a
> > solution to the exact problem you outlined above, that it providing a
> > place for
> > vendors, distros, etc to collaborate on backports and stable branch maint.
> > while following our communities process. Regardless of framing it as being
> > only
> > for drivers it doesn't change that you're talking about the same thing.
> > (this is
> > why in-tree vs out-of-tree was coming up from others, because if it was
> > out-of-tree then you don't have to respect the stable policy)
> >
> ​I don't necessarily fully agree on your statement, I certainly see your
> perspective and can't give an argument to counter it but just that I do see
> a different perspective here as well.  It's not quite so black and white to
> me.  As far as the out of tree comments, I'm not even going to justify
> parts of that thread with a response here.
> 
> >
> > What I was getting at before was if instead of ignoring all the previous
> > conversations we've had about this exact topic (including 2 sessions in
> > Austin)
> >
> ​Sorry, I wasn't a part of the sessions in Austin on the topic of long
> terms support of Cinder drivers.  There's a lot going on during the summits
> these days.
> ​

FWIW, it wasn't just cinder drivers, it was long term support of stable branches,
which includes cinder drivers. Heh, but yeah I can understand that, there is
definitely a lot going on at summit.

> 
> 
> > and people actually stepped up and got involved we wouldn't be having this
> > discussion today. More than likely we'd have enough people to actually
> > maintain
> > a stable branch for longer than we can today. But, instead it feels like
> > our
> > community process for code review and actually testing proposed backports
> > is too
> > much of a burden for any of these parties you've identified to bother
> > getting
> > involved. So we're instead in a situation where we have a proposal to
> > actively
> > circumvent our community policy and procedures for maintaining stable
> > branches.
> >
> ​Yeah, ok... I do see your point here, and as I mentioned I have had this
> conversation with you and others over​
> 
> ​the years and I don't disagree.  I also don't have the ability to "force"
> said parties to do things differently.  So when I try and help customers
> that are having issues my only recourse is an out of tree patch, which then
> when said distro notices or finds out they don't want to support the
> customer any longer based on the code no longer being "their blessed
> code".  The fact is that the distros hold the power in these situations, if
> they happen to own the OS release and the storage then it works out great
> for them, not so much for anybody else.​
> 
> >
> > Creating a free space where vendors can push whatever they want without any
> > gating is not only contrary to our stable branch policy but also goes
> > against
> > some of the fundamental tenants of OpenStack. It's not something I think we
> > should ever be doing.
> >
> ​Not quite sure I agree with that.  We have quite a bit of code that isn't
> gated against, and for years had entire modules and configs that were never
> gated.  I'm not saying that makes it right, or that it's good; but I don't
> think it's quite as dire as it's made out here.​
> 
> 
> >
> > >
> > > If this isn't something we can come to an agreement on as a community,
> > then
> > > I'd suggest we just create our own repo on github outside of upstream and
> > > have it serve the same purpose.
> > >
> >
> > If you're dead set on doing this then I think that's your only recourse,
> > because
> > what you're attempting to do is something outside our community processes
> > and
> > I don't think it has any place in upstream. But, I really think it'd be
> > much
> > better if instead of going off into a corner that all of the people who
> > have
> > complained about the state of our current stable support windows and stable
> > policy actually got involved in this space. That way over time we can make
> > the stable branches something where we can have longer support windows.
> >
> ​I'm certainly not dead set on this, I was hoping that maybe there could be
> some constructive discussion not scolding.
> 
> Yes, in a perfect world we'd have teams that did in fact keep branches
> around longer.  Even then I'm not sure it's realistic.  Things are getting
> better, but traditionally when the distros release updates a year after the
> community does you sort of end up in a pretty crappy position.  Like I
> said, this is getting way better, and who knows... things are catching up
> enough these days that maybe this won't even be an issue any longer in
> another 6 months to  year.
> 
> 
> ​So is the consensus here that the only viable solution is for people to
> invest in keeping the stable branches in general supported longer?  How
> does that work for projects that are interested and have people willing to
> do the work vs projects that don't have the people willing to do the work?
> In other words, Cinder has a somewhat unique problem that Nova, Glance and
> Keystone don't have.  So for Cinder to try and follow the policies,
> processes and philosophies you outlined does that mean that as a project
> Cinder has to try and bend the will of "ALL" of the projects to make this
> happen?  Doesn't seem very realistic to me.​
> 
> Just one last point and I'll move on from the topic.  I'm not sure where
> this illusion that we're testing all the drivers so well is coming from.
> Sure, we require the steps and facade of 3'rd party CI, but dig a bit
> deeper and you soon find that we're not really testing as much as some
> might think here.

I never said our testing coverage was good here, believe me I have no illusions
about that. But, the proposal in this thread would result in having zero testing
in the upstream CI system after we EOL of branch. The infrastructure just
wouldn't be there anymore. It's that point I have a very big issue with, even if
the quality of testing that we would not be running anymore is far from ideal
and mostly just pro forma. What I see as the problem with that is that it means
we would never even try to execute (or install) the code once in the OpenStack
CI before we merge it, despite being part of an OpenStack project. That is what
I see as being contrary to our community standards for doing development.

-Matt Treinish
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160810/e2ef39ae/attachment.pgp>