Open Stack

Thanks everyone for participating and remaining positive and focused
on improving OpenStack. I've posted a review, and I'd like to encourage
everyone to move any future discussion of the Architecture Working group
to that review.

https://review.openstack.org/335141

Excerpts from Clint Byrum's message of 2016-06-17 14:52:43 -0700:
> ar·chi·tec·ture
> ˈärkəˌtek(t)SHər/
> noun
> noun: architecture
> 
>     1. 
> 
>     the art or practice of designing and constructing buildings.
> 
>     synonyms:building design, building style, planning, building, construction; 
> 
>     formalarchitectonics 
> 
>     "modern architecture"
> 
>     the style in which a building is designed or constructed, especially with regard to a specific period, place, or culture.
> 
>     plural noun: architectures
> 
>     "Victorian architecture"
> 
>     2. 
> 
>     the complex or carefully designed structure of something.
> 
>     "the chemical architecture of the human brain"
> 
>     the conceptual structure and logical organization of a computer or computer-based system.
> 
>     "a client/server architecture"
> 
>     synonyms:structure, construction, organization, layout, design, build, anatomy, makeup; 
> 
>     informalsetup 
> 
>     "the architecture of a computer system"
> 
> 
> Introduction
> =========
> 
> OpenStack is a big system. We have debated what it actually is [1],
> and there are even t-shirts to poke fun at the fact that we don't have
> good answers.
> 
> But this isn't what any of us wants. We'd like to be able to point
> at something and proudly tell people "This is what we designed and
> implemented."
> 
> And for each individual project, that is a possibility. Neutron can
> tell you they designed how their agents and drivers work. Nova can
> tell you that they designed the way conductors handle communication
> with API nodes and compute nodes. But when we start talking about how
> they interact with each other, it's clearly just a coincidental mash of
> de-facto standards and specs that don't help anyone make decisions when
> refactoring or adding on to the system.
> 
> Oslo and cross-project initiatives have brought some peace and order
> to the implementation and engineering processes, but not to the design
> process. New ideas still start largely in the project where they are
> needed most, and often conflict with similar decisions and ideas in other
> projects [dlm, taskflow, tooz, service discovery, state machines, glance
> tasks, messaging patterns, database patterns, etc. etc.]. Often times this
> creates a log jam where none of the projects adopt a solution that would
> align with others. Most of the time when things finally come to a head
> these things get done in a piecemeal fashion, where it's half done here,
> 1/3 over there, 1/4 there, and 3/4 over there..., which to the outside
> looks like <architecture> chaos, because that's precisely what it is.
> 
> And this isn't always a technical design problem. OpenStack, for instance,
> isn't really a micro service architecture. Of course, it might look like
> that in diagrams [2], but we all know it really isn't. The compute node is
> home to agents for every single concern, and the API interactions between
> the services is too tightly woven to consider many of them functional
> without the same lockstep version of other services together. A game to
> play is ask yourself what would happen if a service was isolated on its
> own island, how functional would its API be, if at all. Is this something
> that we want? No. But there doesn't seem to be a place where we can go
> to actually design, discuss, debate, and ratify changes that would help
> us get to the point of gathering the necessary will and capability to
> enact these efforts.
> 
> Maybe nova-compute should be isolated from nova, with an API that
> nova, cinder and neutron talk to. Maybe we should make the scheduler
> cross-project aware and capable of scheduling more than just nova
> instances. Maybe we should have experimental groups that can look at how
> some of this functionality could perhaps be delegated to non-openstack
> projects. We hear that Mesos, for example to help with the scheduling
> aspects, but how do we discuss these outside hijacking threads on the
> mailing list? These are things that we all discuss in the hallways
> and bars and parties at the summit, but because they cross projects at
> the design level, and are inherently a lot of social and technical and
> exploratory work, Many of us fear we never get to a place of turning
> our dreams into reality.
> 
> So, with that, I'd like to propose the creation of an Architecture Working
> Group. This group's charge would not be design by committee, but a place
> for architects to share their designs and gain support across projects
> to move forward with and ratify architectural decisions. That includes
> coordinating exploratory work that may turn into being the base of further
> architectural decisions for OpenStack. I would expect that the people
> in this group would largely be senior at the companies involved and,
> if done correctly, they can help prioritize this work by advocating for
> people/fellow engineers to actually make it 'real'. This will give weight
> to specs and implementation changes to make these designs a reality,
> and thus I believe this group would do well to work closely with the
> Oslo Team, where many of the cross-cutting efforts will need to happen.
> 
> How to get involved
> ===================
> 
> If the idea is well received, I'd like to propose a bi-weekly IRC
> meeting at a time convenient for the most interested individuals. I'd
> also like to see a #openstack-architecture channel for gathering real
> time chatter about architecture, and an architecture tag. Finally, I'd
> like to see this group collaborate on direct work using the existing
> openstack-specs repository.
> 
> In closing
> ==========
> 
> First, thanks for reading this whole message and considering this
> evolution of our processes. I am excited to begin this discussion, and
> hope that we can arrive at a plan quickly that leads to us all being
> able to discuss the architecture of OpenStack productively.
> 
> Also, thanks to those of you who helped me review and edit this document.
> 
> [1] http://lists.openstack.org/pipermail/openstack-dev/2016-May/095452.html
> [2] http://docs.openstack.org/admin-guide/_images/openstack-arch-kilo-logical-v1.png