Open Stack

On 09/19/2014 03:29 AM, Thierry Carrez wrote:
> Monty Taylor wrote:
>> I've recently been thinking a lot about Sean's Layers stuff. So I wrote
>> a blog post which Jim Blair and Devananda were kind enough to help me edit.
>>
>> http://inaugust.com/post/108
> 
> Hey Monty,
> 
> As you can imagine, I read that post with great attention. I generally
> like the concept of a tightly integrated, limited-by-design layer #1
> (I'd personally call it "Ring 0") and a large collection of "OpenStack"
> things gravitating around it. That would at least solve the attraction
> of the integrated release, suppress the need for incubation, foster
> competition/innovation within our community, and generally address the
> problem of community scaling. There are a few details on the
> consequences though, and in those as always the devil lurks.
> 
> ## The Technical Committee
> 
> The Technical Committee is defined in the OpenStack bylaws, and is the
> representation of the contributors to "the project". Teams work on code
> repositories, and at some point ask their work to be recognized as part
> of "OpenStack". In doing so, they place their work under the oversight
> of the Technical Committee. In return, team members get to participate
> in electing the technical committee members (they become "ATC"). It's a
> balanced system, where both parties need to agree: the TC can't force
> itself as overseer of a random project, and a random project can't just
> decide by itself it is "OpenStack".
> 
> I don't see your proposal breaking that balanced system, but it changes
> its dynamics a bit. The big tent would contain a lot more members. And
> while the TC would arguably bring a significant share of its attention
> to Ring 0, its voters constituency would mostly consist of developers
> who do not participate in Ring 0 development. I don't really see it as
> changing dramatically the membership of the TC, but it's a consequence
> worth mentioning.

Agree. I'm willing to bet it'll be better not worse to have a large
constituency - but it's also problem that it's a giant disaster. I'm
still on board with going for it.

> ## Programs
> 
> Programs were created relatively recently as a way to describe which
> teams are "in OpenStack" vs. which ones aren't. They directly tie into
> the ATC system: if you contribute to code repositories under a blessed
> program, then you're an ATC, you vote in TC elections and the TC has
> some oversight over your code repositories. Previously, this was granted
> at a code repository level, but that failed to give flexibility for
> teams to organize their code in the most convenient manner for them. So
> we started to bless teams rather than specific code repositories.
> 
> Now, that didn't work out so well. Some programs were a 'theme', like
> Infrastructure, or Docs. For those, competing efforts do not really make
> sense: there can only be one, and competition should happen inside those
> efforts rather than outside. Some programs were a 'team', like
> Orchestration/Heat or Deployment/TripleO. And that's where the model
> started to break: some new orchestration things need space, but the
> current Heat team is not really interested in maintaining them. What's
> the point of being under the same program then ? And TripleO is not the
> only way to deploy OpenStack, but its mere existence (and name)
> prevented other flowers to bloom in our community.
> 
> You don't talk much about programs in your proposal. In particular, you
> only mention layer 1, "Cloud Native" applications, "User Interface"
> applications, and "Operator" applications. So I'm unsure of where, if
> anywhere, would "Infrastructure" or "Docs" repositories live.
> 
> Here is how I see it could work. We could keep 'theme' programs (think
> Infra, Release cycle management, Docs, QA) with their current structure
> (collection of code repositories under a single team/PTL). We would get
> rid of 'team' programs completely, and just have a registry of
> "OpenStack" code repositories (openstack*/*, big tent). Each of those
> could have a specific PTL, or explicitely inherit its PTL from another
> code repository. Under that PTL, they could have separate or same core
> review team, whatever maps reality and how they want to work (so we
> could say that openstack/python-novaclient inherits its PTL from
> openstack/nova and doesn't need a specific one). That would allow us to
> map anything that may come in. Oslo falls a bit in between, could be
> considered a 'theme' program or several repos sharing PTL.

I think we can do what you're saying and generalize a little bit. What
if we declared programs, as needed, when we think there is a need to
"pick a winner". (I think we can all agree that early winner picking is
an unintended but very real side effect of the current system)

And when I say "need to" - I mean it in the same sense as "Production
Ready"  The themes you listed are excellent ones - it makes no sense to
have two Infras, two QAs or two Docs teams. On the other hand, maybe
someone else wants to take a stab at the general problem space that
congress and manilla and heat are all playing in. Do we need to bless
that yet? No. Are those people all honestly trying to solve problems for
OpenStack? Absolutely!

It's a judgement call for sure - but I like the idea of keeping the
programs for some things and not for others.

> ## The release and the development cycle
> 
> You touch briefly on the consequences of your model for the "common
> release" and our development cycle. Obviously we would release the ring
> 0 projects in an integrated manner on a time-based schedule.
> 
> For the other projects, we have a choice:
> 
> - the "ring 0" model (development milestones, coordinated final release)
> - the "Swift" model (release as-needed, coordinated final release)
> - the "Oslo" model (release as-needed, try to have one final one before
> end of cycle)
> - the "Client libraries" model (release as-needed)
> 
> If possible, I would like to avoid the "Swift" model, which is the most
> costly from a release management standpoint. All projects following the
> ring 0 model are easy to keep track of, using common freezes etc. So
> it's easy to make sure they will be ready in time for the coordinated
> release. Each project following the Swift model would be a special case,
> and that adds up to a lot of load on the release management team.
> Keeping track of one project doing that is OK. 5 or 20, not so much. So
> I'd advise we only keep ring0, Oslo and client lib models as options.
> Release management would just care about ring 0, and provide tools and
> advice for all the others, but not being responsible for them.

I'm in.

> What about the development cycle ? I think it's part of our "identity":
> being "OpenStack" is also about having the same notion of passing time,
> the same reference points. So I think it would probably be a good idea,
> even for projects that purely release "as-needed", to organize their
> development at least vaguely around the notion of a common cycle.

For what it's worth, Infra even tries to do freezes around release (this
concept is known as "don't land any of Monty's patches for a week around
major freezes)

I think it's worth having a more expansive discussion around the release
cycle overall, but this is enough massive change for me for now.

> ## The design summit
> 
> I'm not a big fan of your #11 suggestion. I guess I just don't want to
> defer all in-project alignment and work to mid-cycle meetups, and force
> all contributors to travel all the time. I think the right time to reach
> alignment on the goals is at the start of a development cycle, not in
> the middle of it. The middle of a cycle is a good time to get things
> done. So I would still like teams to be able to discuss their goals at
> the Design Summit, at the start of a cycle.
> 
> Now I realize this creates a pretty significant issue, since Design
> Summit space is one resource which can't really support the "infinite
> tent" model. My proposal to solve it would be to give ample space to
> cross-project issues, ring 0 projects, and 'theme' programs. Everyone
> else gets limited space, which may boil down to a pod. Then we work on
> options, so that whoever who wants to organize a large meetup in
> parallel with the summit may find space to do so in neighboring hotels.
> 
> The main value in that proposal is that it makes the "Design Summit"
> space needs more predictable. I've been visiting spaces for the 2016 and
> 2018 summits recently -- it's really difficult to make plans when you
> don't know how many projects you'll have to host. And it's really
> difficult to get good locations and times (and prices) if you don't book
> at least 2 years in advance.

++

My main point was that we should optimize summit time for things that we
can only do then - gnarly cross-project issues being one of the biggest
examples. The other was that it would be great to add a bit more topic
malleability to be able to respond to things we hear in the operator
feedback sessions - especially in the heavily-contested cross-project
time. (saying "just use pods" for those emergent issues isn't a workable
thing because we likely need people who are double booked)

But in general, yeah.

> ## The Foundation bylaws
> 
> There are a number of terms used in the bylaws, and we may need to map
> our new structure to it, make sure it doesn't require a bylaws change.
> I didn't spot any showstopper yet. The "integrated release" could be the
> ring 0 (that would mean that defcore would only get to pick between
> Nova/Neutron/Cinder/Keystone/Designate for the trademark programs). The
> "OpenStack project" could be the big tent. We keep the TC, the ATCs
> concepts the same.

I don't think ring 0 -> bylaws "Integrated" is the right mapping. I just
spent a board meeting yelling about how I'm pretty sure that any logical
system that somehow winds up leaving swift out has a flaw, and I think
that applies here too.

I don't have a great answer - but you're right - we need a name mapping.

OTOH, the board was considering a bylaws change for defcore - perhaps we
should take a quick pass at proposing a bylaws change to make the
language a little bit more general and get us to the point of the
_intent_ of that - which is that defcore can pick from at most a list of
projects suggested by the TC.

OR - we could go massively the other way WRT the bylaws, and we could
consider _every_ crazy big-tent thing under our care "in the integrated
release" - and count on the fact that _probably_ defcore is not even
going to think about including something in their definition until we've
blessed something as production ready ...

> 
> That was a long answer, but then so was your original post :P

We should do this more often ... ;)