Open Stack

On Tue, Aug 12, 2014 at 12:30 AM, Joe Gordon <joe.gordon0 at gmail.com> wrote:

>
>
>
> On Fri, Aug 8, 2014 at 6:58 AM, Kyle Mestery <mestery at mestery.com> wrote:
>
>> On Thu, Aug 7, 2014 at 1:26 PM, Joe Gordon <joe.gordon0 at gmail.com> wrote:
>> >
>> >
>> >
>> > On Tue, Aug 5, 2014 at 9:03 AM, Thierry Carrez <thierry at openstack.org>
>> > wrote:
>> >>
>> >> Hi everyone,
>> >>
>> >> With the incredible growth of OpenStack, our development community is
>> >> facing complex challenges. How we handle those might determine the
>> >> ultimate success or failure of OpenStack.
>> >>
>> >> With this cycle we hit new limits in our processes, tools and cultural
>> >> setup. This resulted in new limiting factors on our overall velocity,
>> >> which is frustrating for developers. This resulted in the burnout of
>> key
>> >> firefighting resources. This resulted in tension between people who try
>> >> to get specific work done and people who try to keep a handle on the
>> big
>> >> picture.
>> >>
>> >> It all boils down to an imbalance between strategic and tactical
>> >> contributions. At the beginning of this project, we had a strong inner
>> >> group of people dedicated to fixing all loose ends. Then a lot of
>> >> companies got interested in OpenStack and there was a surge in
>> tactical,
>> >> short-term contributions. We put on a call for more resources to be
>> >> dedicated to strategic contributions like critical bugfixing,
>> >> vulnerability management, QA, infrastructure... and that call was
>> >> answered by a lot of companies that are now key members of the
>> OpenStack
>> >> Foundation, and all was fine again. But OpenStack contributors kept on
>> >> growing, and we grew the narrowly-focused population way faster than
>> the
>> >> cross-project population.
>> >>
>> >>
>> >> At the same time, we kept on adding new projects to incubation and to
>> >> the integrated release, which is great... but the new developers you
>> get
>> >> on board with this are much more likely to be tactical than strategic
>> >> contributors. This also contributed to the imbalance. The penalty for
>> >> that imbalance is twofold: we don't have enough resources available to
>> >> solve old, known OpenStack-wide issues; but we also don't have enough
>> >> resources to identify and fix new issues.
>> >>
>> >> We have several efforts under way, like calling for new strategic
>> >> contributors, driving towards in-project functional testing, making
>> >> solving rare issues a more attractive endeavor, or hiring resources
>> >> directly at the Foundation level to help address those. But there is a
>> >> topic we haven't raised yet: should we concentrate on fixing what is
>> >> currently in the integrated release rather than adding new projects ?
>> >
>> >
>> > TL;DR: Our development model is having growing pains. until we sort out
>> the
>> > growing pains adding more projects spreads us too thin.
>> >
>> +100
>>
>> > In addition to the issues mentioned above, with the scale of OpenStack
>> today
>> > we have many major cross project issues to address and no good place to
>> > discuss them.
>> >
>> We do have the ML, as well as the cross-project meeting every Tuesday
>> [1], but we as a project need to do a better job of actually bringing
>> up relevant issues here.
>>
>> [1] https://wiki.openstack.org/wiki/Meetings/ProjectMeeting
>>
>> >>
>> >>
>> >> We seem to be unable to address some key issues in the software we
>> >> produce, and part of it is due to strategic contributors (and core
>> >> reviewers) being overwhelmed just trying to stay afloat of what's
>> >> happening. For such projects, is it time for a pause ? Is it time to
>> >> define key cycle goals and defer everything else ?
>> >
>> >
>> >
>> > I really like this idea, as Michael and others alluded to in above, we
>> are
>> > attempting to set cycle goals for Kilo in Nova. but I think it is worth
>> > doing for all of OpenStack. We would like to make a list of key goals
>> before
>> > the summit so that we can plan our summit sessions around the goals. On
>> a
>> > really high level one way to look at this is, in Kilo we need to pay
>> down
>> > our technical debt.
>> >
>> > The slots/runway idea is somewhat separate from defining key cycle
>> goals; we
>> > can be approve blueprints based on key cycle goals without doing slots.
>>  But
>> > with so many concurrent blueprints up for review at any given time, the
>> > review teams are doing a lot of multitasking and humans are not very
>> good at
>> > multitasking. Hopefully slots can help address this issue, and hopefully
>> > allow us to actually merge more blueprints in a given cycle.
>> >
>> I'm not 100% sold on what the slots idea buys us. What I've seen this
>> cycle in Neutron is that we have a LOT of BPs proposed. We approve
>> them after review. And then we hit one of two issues: Slow review
>> cycles, and slow code turnaround issues. I don't think slots would
>> help this, and in fact may cause more issues. If we approve a BP and
>> give it a slot for which the eventual result is slow review and/or
>> code review turnaround, we're right back where we started. Even worse,
>> we may have not picked a BP for which the code submitter would have
>> turned around reviews faster. So we've now doubly hurt ourselves. I
>> have no idea how to solve this issue, but by over subscribing the
>> slots (e.g. over approving), we allow for the submissions with faster
>> turnaround a chance to merge quicker. With slots, we've removed this
>> capability by limiting what is even allowed to be considered for
>> review.
>>
>
> Slow review: by limiting the number of blueprints up we hope to focus our
> efforts on fewer concurrent things
> slow code turn around: when a blueprint is given a slot (runway) we will
> first make sure the author/owner is available for fast code turnaround.
>
> If a blueprint review stalls out (slow code turnaround, stalemate in
> review discussions etc.) we will take the slot and give it to another
> blueprint.
>

How is that more efficient than today's do-the-best-we-can approach? It
just sounds like bureaucracy to me.

Reading between the lines throughout this thread, it sounds like what we're
lacking is a reliable method to communicate review prioritization to core
reviewers.

What the community wants/needs is just noise generated via IRC, email,
etherpad, launchpad, etc. It's part of a PTL's job to help filter that
noise and help communicate that back to core reviewers, but that just ends
up creating more IRC/email/etc, rather than directly impacting anyone's
experience with gerrit.


>
>>
>> Thanks,
>> Kyle
>>
>> >>
>> >>
>> >> On the integrated release side, "more projects" means stretching our
>> >> limited strategic resources more. Is it time for the Technical
>> Committee
>> >> to more aggressively define what is "in" and what is "out" ? If we go
>> >> through such a redefinition, shall we push currently-integrated
>> projects
>> >> that fail to match that definition out of the "integrated release"
>> inner
>> >> circle ?
>> >>
>> >> The TC discussion on what the integrated release should or should not
>> >> include has always been informally going on. Some people would like to
>> >> strictly limit to end-user-facing projects. Some others suggest that
>> >> "OpenStack" should just be about integrating/exposing/scaling smart
>> >> functionality that lives in specialized external projects, rather than
>> >> trying to outsmart those by writing our own implementation. Some others
>> >> are advocates of carefully moving up the stack, and to resist from
>> >> further addressing IaaS+ services until we "complete" the pure IaaS
>> >> space in a satisfactory manner. Some others would like to build a
>> >> roadmap based on AWS services. Some others would just add anything that
>> >> fits the incubation/integration requirements.
>> >>
>> >>
>> >> On one side this is a long-term discussion, but on the other we also
>> >> need to make quick decisions. With 4 incubated projects, and 2 new ones
>> >> currently being proposed, there are a lot of people knocking at the
>> door.
>> >
>> >
>> >
>> > I have a slightly different short term opinion on what 'OpenStack'
>> should
>> > be: it should work really well.
>> >
>> > While we need to figure out howto increase our strategic resources,
>> > realistically I think that will still be the limiting factor in Kilo.
>> So in
>> > order to better allocate our cross project strategic resources, I think
>> we
>> > should do a project level triage ('identify projects that are likely to
>> die,
>> > regardless of what care they receive' to paraphrase the medical
>> definition
>> > of the term), and tighten our focus until we get our development process
>> > into a better state.
>> >
>> >
>> >>
>> >>
>> >> Thanks for reading this braindump this far. I hope this will trigger
>> the
>> >> open discussions we need to have, as an open source project, to reach
>> >> the next level.
>> >
>> >
>> > Thank you for bringing this topic up.
>> >
>> >>
>> >>
>> >> Cheers,
>> >>
>> >> --
>> >> Thierry Carrez (ttx)
>> >>
>> >> _______________________________________________
>> >> OpenStack-dev mailing list
>> >> OpenStack-dev at lists.openstack.org
>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>> >
>> >
>> > _______________________________________________
>> > OpenStack-dev mailing list
>> > OpenStack-dev at lists.openstack.org
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140812/4bfb902f/attachment-0001.html>