Open Stack

On 20/06/16 11:31 -0700, Clint Byrum wrote:
>Excerpts from Joshua Harlow's message of 2016-06-17 15:33:25 -0700:
>> Thanks for getting this started Clint,
>>
>> I'm happy and excited to be involved in helping try to guide the whole
>> ecosystem together (it's also why I like being in oslo) to a
>> architecture that is more cohesive (and is more of something that we can
>> say to our current or future children that we were all involved and
>> proud to be involved in creating/maturing...).
>>
>> At a start, for said first meeting, any kind of agenda come to mind, or
>> will it be more a informal gathering to start (either is fine with me)?
>>
>
>I've been hesitant to fill this in too much as I'm still forming the
>idea, but here are the items I think are most compelling to begin with:
>
>* DLM's across OpenStack -- This is already under way[1], but it seems to
>  have fizzled out. IMO that is because there's no working group who
>  owns it. We need to actually write some plans.
>
>* Messaging patterns -- There was a recent discussion about nuances in
>  oslo.messaging's implementation that vary driver to driver. I'd like to
>  make sure we have all of the ways messaging is used written down and
>  make sure groups have guidance on how each one should (or shouldn't)
>  be used, including documentation of anti-patterns, and a plan for
>  simplifying communication between processes in OpenStack where
>  possible.
>
>* True Microservices -- OpenStack's services are heavily
>  interdependent. It makes it hard for an engineer to make an impact
>  without becoming an expert on all of them, and it also leads to a heavy
>  burden on operators and QA as they end up having to debug the system
>  as a whole. We should write down how the system is intertwined now, and
>  develop plans for unwinding that and allowing each service to stand on
>  its own, including having stronger testing in isolation. (This includes
>  exploring the thought that nova-compute could become its own project
>  that all of the other things communicate with over well defined APIs).
>
>These could keep a small group of architects and engineers busy for a
>year or more. I'm sure others have things they'd like to design and
>improve in OpenStack at this level.
>
>Once we have broad agreement on such a group, and perhaps some guidance
>from the TC, we can propose an agenda and prioritize efforts as part of
>the first few meetings.

As you mentioned, these are topics that will keep a group of folks busy for a
year or more. Would it make sense to just pick one for now as a starting task
for this group and see how things evolve?

Flavio

>[1] http://specs.openstack.org/openstack/openstack-specs/specs/chronicles-of-a-dlm.html
>> -Josh
>>
>> Clint Byrum wrote:
>> > ar·chi·tec·ture
>> > ˈärkəˌtek(t)SHər/
>> > noun
>> > noun: architecture
>> >
>> >      1.
>> >
>> >      the art or practice of designing and constructing buildings.
>> >
>> >      synonyms:building design, building style, planning, building, construction;
>> >
>> >      formalarchitectonics
>> >
>> >      "modern architecture"
>> >
>> >      the style in which a building is designed or constructed, especially with regard to a specific period, place, or culture.
>> >
>> >      plural noun: architectures
>> >
>> >      "Victorian architecture"
>> >
>> >      2.
>> >
>> >      the complex or carefully designed structure of something.
>> >
>> >      "the chemical architecture of the human brain"
>> >
>> >      the conceptual structure and logical organization of a computer or computer-based system.
>> >
>> >      "a client/server architecture"
>> >
>> >      synonyms:structure, construction, organization, layout, design, build, anatomy, makeup;
>> >
>> >      informalsetup
>> >
>> >      "the architecture of a computer system"
>> >
>> >
>> > Introduction
>> > =========
>> >
>> > OpenStack is a big system. We have debated what it actually is [1],
>> > and there are even t-shirts to poke fun at the fact that we don't have
>> > good answers.
>> >
>> > But this isn't what any of us wants. We'd like to be able to point
>> > at something and proudly tell people "This is what we designed and
>> > implemented."
>> >
>> > And for each individual project, that is a possibility. Neutron can
>> > tell you they designed how their agents and drivers work. Nova can
>> > tell you that they designed the way conductors handle communication
>> > with API nodes and compute nodes. But when we start talking about how
>> > they interact with each other, it's clearly just a coincidental mash of
>> > de-facto standards and specs that don't help anyone make decisions when
>> > refactoring or adding on to the system.
>> >
>> > Oslo and cross-project initiatives have brought some peace and order
>> > to the implementation and engineering processes, but not to the design
>> > process. New ideas still start largely in the project where they are
>> > needed most, and often conflict with similar decisions and ideas in other
>> > projects [dlm, taskflow, tooz, service discovery, state machines, glance
>> > tasks, messaging patterns, database patterns, etc. etc.]. Often times this
>> > creates a log jam where none of the projects adopt a solution that would
>> > align with others. Most of the time when things finally come to a head
>> > these things get done in a piecemeal fashion, where it's half done here,
>> > 1/3 over there, 1/4 there, and 3/4 over there..., which to the outside
>> > looks like<architecture>  chaos, because that's precisely what it is.
>> >
>> > And this isn't always a technical design problem. OpenStack, for instance,
>> > isn't really a micro service architecture. Of course, it might look like
>> > that in diagrams [2], but we all know it really isn't. The compute node is
>> > home to agents for every single concern, and the API interactions between
>> > the services is too tightly woven to consider many of them functional
>> > without the same lockstep version of other services together. A game to
>> > play is ask yourself what would happen if a service was isolated on its
>> > own island, how functional would its API be, if at all. Is this something
>> > that we want? No. But there doesn't seem to be a place where we can go
>> > to actually design, discuss, debate, and ratify changes that would help
>> > us get to the point of gathering the necessary will and capability to
>> > enact these efforts.
>> >
>> > Maybe nova-compute should be isolated from nova, with an API that
>> > nova, cinder and neutron talk to. Maybe we should make the scheduler
>> > cross-project aware and capable of scheduling more than just nova
>> > instances. Maybe we should have experimental groups that can look at how
>> > some of this functionality could perhaps be delegated to non-openstack
>> > projects. We hear that Mesos, for example to help with the scheduling
>> > aspects, but how do we discuss these outside hijacking threads on the
>> > mailing list? These are things that we all discuss in the hallways
>> > and bars and parties at the summit, but because they cross projects at
>> > the design level, and are inherently a lot of social and technical and
>> > exploratory work, Many of us fear we never get to a place of turning
>> > our dreams into reality.
>> >
>> > So, with that, I'd like to propose the creation of an Architecture Working
>> > Group. This group's charge would not be design by committee, but a place
>> > for architects to share their designs and gain support across projects
>> > to move forward with and ratify architectural decisions. That includes
>> > coordinating exploratory work that may turn into being the base of further
>> > architectural decisions for OpenStack. I would expect that the people
>> > in this group would largely be senior at the companies involved and,
>> > if done correctly, they can help prioritize this work by advocating for
>> > people/fellow engineers to actually make it 'real'. This will give weight
>> > to specs and implementation changes to make these designs a reality,
>> > and thus I believe this group would do well to work closely with the
>> > Oslo Team, where many of the cross-cutting efforts will need to happen.
>> >
>> > How to get involved
>> > ===================
>> >
>> > If the idea is well received, I'd like to propose a bi-weekly IRC
>> > meeting at a time convenient for the most interested individuals. I'd
>> > also like to see a #openstack-architecture channel for gathering real
>> > time chatter about architecture, and an architecture tag. Finally, I'd
>> > like to see this group collaborate on direct work using the existing
>> > openstack-specs repository.
>> >
>> > In closing
>> > ==========
>> >
>> > First, thanks for reading this whole message and considering this
>> > evolution of our processes. I am excited to begin this discussion, and
>> > hope that we can arrive at a plan quickly that leads to us all being
>> > able to discuss the architecture of OpenStack productively.
>> >
>> > Also, thanks to those of you who helped me review and edit this document.
>> >
>> > [1] http://lists.openstack.org/pipermail/openstack-dev/2016-May/095452.html
>> > [2] http://docs.openstack.org/admin-guide/_images/openstack-arch-kilo-logical-v1.png
>> >
>> > __________________________________________________________________________
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>__________________________________________________________________________
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-- 
@flaper87
Flavio Percoco
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160622/33693314/attachment.pgp>