Open Stack

On Thu, Sep 25, 2014 at 3:55 AM, Zane Bitter <zbitter at redhat.com> wrote:

> On 18/09/14 14:53, Monty Taylor wrote:
>
>> Hey all,
>>
>> I've recently been thinking a lot about Sean's Layers stuff. So I wrote
>> a blog post which Jim Blair and Devananda were kind enough to help me
>> edit.
>>
>> http://inaugust.com/post/108
>>
>
> Thanks Monty, I think there are some very interesting ideas in here.
>
> I'm particularly glad to see the 'big tent' camp reasserting itself,
> because I have no sympathy with anyone who wants to join the OpenStack
> community and then bolt the door behind them. Anyone who contributes to a
> project that is related to OpenStack's goals, is willing to do things the
> OpenStack way, and submits itself to the scrutiny of the TC deserves to be
> treated as a member of our community with voting rights, entry to the
> Design Summit and so on.
>
> I'm curious how you're suggesting we decide which projects satisfy those
> criteria though. Up until now, we've done it through the incubation process
> (or technically, the new program approval process... but in practice we've
> never added a project that was targeted for eventual inclusion in the
> integrated release to a program without incubating it). Would the TC
> continue to judge whether a project is doing things the OpenStack way prior
> to inclusion, or would we let projects self-certify? What does it mean for
> a project to submit itself to TC scrutiny if it knows that realistically
> the TC will never have time to actually scrutinise it? Or are you not
> suggesting a change to the current incubation process, just a willingness
> to incubate multiple projects in the same problem space?
>
> I feel like I need to play devil's advocate here, because overall I'm just
> not sure I understand the purpose of arbitrarily - and it *is* arbitrary -
> declaring "Layer #1" to be anything required to run Wordpress. To anyone
> whose goal is not to run Wordpress, how is that relevant?
>
> Speaking of arbitrary, I had to laugh a little at this bit:
>
>  Also, please someone notice that the above is too many steps and should
> be:
>
>   openstack boot gentoo on-a 2G-VM with-a publicIP with-a 10G-volume
> call-it blog.inaugust.com
>
> That's kinda sorta exactly what Heat does ;) Minus the part about assuming
> there is only one kind of application, obviously.
>
>
:-)


>
> I think there are a number of unjustified assumptions behind this
> arrangement of things. I'm going to list some here, but I don't want anyone
> to interpret this as a personal criticism of Monty. The point is that we
> all suffer from biases - not for any questionable reasons but purely as a
> result of our own experiences, who we spend our time talking to and what we
> spend our time thinking about - and therefore we should all be extremely
> circumspect about trying to bake our own mental models of what OpenStack
> should be into the organisational structure of the project itself.
>
> * Assumption #1: The purpose of OpenStack is to provide a Compute cloud
>
> This assumption is front-and-centre throughout everything Monty wrote. Yet
> this wasn't how the OpenStack project started. In fact there are now at
> least three services - Swift, Nova, Zaqar - that could each make sense as
> the core of a standalone product.
>

Agree.


>
> Yes, it's true that Nova effectively depends on Glance and Neutron (and
> everything depends on Keystone). We should definitely document that
> somewhere. But why does it make Nova special?
>
> * Assumption #2: Yawnoc's Law
>
> Don't bother Googling that, I just made it up. It's the reverse of
> Conway's Law:
>
>   Infra engineers who design governance structures for OpenStack are
>   constrained to produce designs that are copies of the structure of
>   Tempest.
>
> I just don't understand why that needs to be the case. Currently, for
> understandable historic reasons, every project gates against every other
> project. That makes no sense any more, completely independently of the
> project governance structure. We should just change it! There is no
> organisational obstacle to changing how gating works.
>
> Even this proposal doesn't entirely make sense on this front - e.g.
> Designate requires only Neutron and Keystone... why should Nova, Glance and
> every other project in "Layer 1" gate against it, and vice-versa?
>
> I suggested in another thread[1] a model where each project would publish
> a set of tests, each project would decide which sets of tests to pull in
> and gate on, and Tempest would just be a shell for setting up the
> environment and running the selected tests. Maybe that idea is crazy or at
> least needs more work (it certainly met with only crickets and tumbleweeds
> on the mailing list), but implementing it wouldn't require TC intervention
> and certainly not by-laws changes. It just requires... implementing it.
>
> Perhaps the idea here is that by designating "Layer 1" the TC is
> indicating to projects which other projects they should accept gate test
> jobs from (a function previously fulfilled by Incubation). I'd argue that
> this is a very bad way to do it, because (a) it says nothing to projects
> outside of "Layer 1" how they should decide, and (b) it jumps straight to
> the TC mandating the result without even letting the projects try to sort
> it out amongst themselves.
>
> For example, I would actually prefer that Nova not gate against Heat
> because Nova is pretty unlikely to break us and the trade-off of putting us
> in a position to accidentally break them is not worth it. No edict from the
> TC required. On the other hand, I would push very strongly for all of the
> python-*client libraries to gate against both Heat and Horizon, because
> they can easily break us - and if they break us, they're probably breaking
> other users out there too, so I'm confident I could convince people that
> this would be mutually beneficial. (It could potentially even extend so far
> as running the unit tests of Heat and Horizon in the client gates, to avoid
> issues like [2].)
>
> [1] http://lists.openstack.org/pipermail/openstack-dev/2014-
> September/045446.html
> [2] http://lists.openstack.org/pipermail/openstack-dev/2014-
> September/046686.html
>
> * Assumption #3: The world is static
>
> This is a giant red flag:
>
>   "the set of things in Layer #1 should never change -- unless we
>    refactor something already in Layer #1 into a new project."
>

I think this Layer #1 thing needs to die (as something different/special,
not necessarily as a grouping of projects).


>
> There is no greater act of hubris than to stick a stake in the ground and
> declare that "we will never know more than we do at this moment; we'll only
> get dumber from here, so we must precommit to all of our future decisions
> based on the information we have at present".
>
> What if, for example, Nova wanted to add a dependency on Zaqar? They'd be
> prevented from doing so because Zaqar is not used by Wordpress. How is that
> relevant? A rigid ban on dependencies is a death knell for innovation.
>
> Can you really never imagine a time where it might be better to run
> Wordpress on a container service rather than a full-fledged VM? I guess
> that's OK but only as long as it starts in Nova and then gets split out?
> Because... nova-core don't have enough to do?
>
> And none of this is any help at all to projects outside of "Layer 1",
> because they get no guidance at all on what makes sense to depend on. This
> is already hurting with our current system (for example, Mistral is
> implementing a bunch of notification stuff that should properly be
> delegated to Zaqar, and in fact as of 6 months ago it was the centrepiece
> of the design), and the TC abdicating all interest in the subject will make
> it even worse.
>
> * Assumption #4: The sky is falling
>
> From reading openstack-dev, it's pretty clear that both the QA and Nova
> programs are facing a scaling crisis of sorts. It's easy to see why anybody
> deeply involved with either or both of those two would indeed think that
> radical change is required. I'm not sure, however, that the same sense of
> crisis pervades all of the other projects. We all have a lot of work to do,
> but I suspect that most projects would say that they are trucking along
> nicely. Meanwhile, the proposal is to change pretty much everything about
> how OpenStack is organised *except* QA and Nova (in fact, it creates
> incentives to stick even more stuff inside Nova), which remain sacrosanct.
> That doesn't seem like attacking the problem at its source.
>
>
> So we've identified the minimum set of OpenStack services required to
> sensibly run Wordpress. Awesome! Somebody should totally write a blog post
> about that. But officially and permanently baking that in as the structure
> of the OpenStack project? I hate to use the c-word, but the bottom line is
> that "Layer 1" just resurrects Core with a pretext to finally kick Swift
> out. That seems particularly ironic, because I would pay good money to be a
> fly on the wall in a board meeting where anyone but Monty proposed such a
> thing in those terms, just to watch his reaction. Given that the TC
> informed the DefCore committee that it regarded everything that has
> graduated to the integrated release as the "designated sections" for
> DefCore purposes and told them to go do their own dirty work, you can bet
> your last dollar that this will be interpreted as a TC endorsement for
> permanently excluding Swift - and all the other non-"Layer 1" projects -
> from the designated sections. In fact, by removing only those tests from
> Tempest it's likely to have the side-effect of eliminating them from
> RefStack altogether.
>
>
> Let's sum up, first by looking at a list of questions that developers,
> distributors, operators and users might ask about a project:
>
> 1) Are they "one of us"?
> 2) Should I gate against it?
> 3) Can I add a dependency on it?
> 4) Should this be widely distributed as part of OpenStack?
> 5) Can I use this knowing that the API will be somewhat stable?
> 6) Should this be used at scale in production?
>
>
> Here's how the TC is answering those questions at the moment:
>
> 1) New program acceptance + incubation or adoption processes
> 2) Incubation process
> 3) Graduation process
> 4) Graduation process
> 5) Graduation process
> 6) You're on your own
>
> Here's Monty's answers:
>
> 1) ???
> 2) No
> 3) No
> 4) You're on your own
> 5) You're on your own?
> 6) "CERN test"
>
> Both of those feel unsatisfactory in different ways. Monty's suggestions
> seem like an overly radical change to me; I would like to try something a
> bit more incremental to give us the chance to see how the community adapts:
>
> 1) Incubation process (much lower bar)
> 2) Do your own cost/benefit analysis
> 3) Graduation process
> 4) Graduation process (maintain high bar, but less capricious)
> 5) Graduation process
> 6) TC/UC production-readiness review
>
>
> Finally, since the motivation for change is that we think the current
> structure isn't scaling, let's examine the individual things that are
> currently pain points:
>
> * Continuous Integration
>
> We all agree that the gate doesn't scale. I submit that it doesn't scale
> because it tests every project against every other project, and that
> kicking projects out of the gate not only fails to solve the problem in the
> long term (since the projects that _are_ in will continue to grow), but
> also ignores the actual risks that the gate is meant to guard against in
> favour of an arbitrary designation.
>
> We should scale the gate by only gating projects against other projects
> where the benefit in reduced risk outweighs the cost in increased risk of
> false negatives. For projects that don't depend on each other at all, the
> benefit is precisely zero (beyond the install-only gate suggested by Monty,
> which I support). We should apply the same cost-benefit calculation
> regardless of how involved the projects in question are with running
> Wordpress, and we should let projects themselves decide what to gate
> against in the first instance, with the TC only stepping in in the event
> that consensus can't be reached by other means.
>
> * Documentation
>
> This is a tricky one, and not an area of OpenStack that I am an expert on.
> It does seem to me that the only real solution is to make projects more
> responsible for their own documentation. Arbitrarily splitting projects
> into a category where they're not responsible at all and a category where
> they're completely on their own doesn't seem like a good solution.
>
> * Release Management
>
> This is something we have not really even attempted to scale beyond
> Thierry. As a first step, there is no real organisational obstacle to
> having a different release manager for incubated projects than for
> integrated projects, it's more a matter of making it known to either the
> Foundation or the various companies who employ contributors that we need
> one. I don't want to make that process sound trivial, but I'm confident
> that the release management program could handle it, and I think we should
> at least give them a chance to try before pre-emptively kicking anything
> non-Wordpress-related out of the release forever.
>
> * Technical Committee
>
> It is inevitable that we will reach a point where the Technical Committee
> itself does not scale. I'm surprised, because I thought that was a ways
> off, but after watching the latest Zaqar fiasco I think we have to consider
> the possibility that we have reached that point already.
>
> Perhaps we should consider having subcommittees, maybe based on the
> groupings identified by John (Dickinson), possibly comprised of the
> relevant PTLs plus a representative of the TC. These subcommittees would do
> the legwork of investigating new projects making their way through the
> incubation/graduation process and report summaries and recommendations to
> the TC.
>
>
I really liked the larger grouping of programs (can't remember who
suggested it), but I think this is the way to go.
I don't want to get  into bike shedding on what the names are and what
projects go into those groups, but...

I think each of those groups needs their own TC/docs/relmgmt/QA (TC could
be just the ptl's - yes back to that).
Basically to scale better as an organization we need each of those groups
to be more standalone.


As far as "what should an operator deploy", I like the mobile setup where
you have the basics plus a mechanism to install more services (app store).
I think this fits nicely into the "market place" language and allowing more
projects to compete.

So operators deploy what ever the "service store" needs and that can deploy
a range of optional OpenSource extras. Then you could also get
commercial bits too (if you wanted).

Basically make use of a paas of some form, and we package optional services
to be deployed by that.
(also it would be good to have tenant based services/endpoints)

-Angus


> cheers,
> Zane.
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140925/95e44c9b/attachment.html>