Open Stack

Qiming Teng wrote:
> On Mon, Jan 16, 2017 at 08:21:02PM +0000, Fox, Kevin M wrote:
>> IMO, This is why the big tent has been so damaging to OpenStack's progress. Instead of lifting the commons up, by requiring dependencies on other projects, there by making them commonly deployed and high quality, post big tent, each project reimplements just enough to get away with making something optional, and then the commons, and OpenStack as a whole suffers. This behavior MUST STOP if OpenStack is to make progress again. Other projects, such as Kubernetes are making tremendous progress because they are not hamstrung by one component trying desperately not to depend on another when the dependency is appropriate. They enhance the existing component until its suitable and the whole project benefits. Yes, as an isolated dev, the behavior to make deps optional seems to make sense. But as a whole, OpenStack is suffering and will become increasingly irrelevant moving forward if the current path is continued. Please, please reconsider what the current stance on dependencies is d
>>  oing to the community. This problem is not just isolated to barbican, but lots of other projects as well. We can either help pull each other up, or we can step on each other to try and get "on top". I'd rather we help each other rather then the destructive path we seem to be on.
> 
> Very well said, Kevin. The problem is not just about Barbican. Time for
> the TC and the whole community to rethink or even just to realize
> where we are heading ... Time for each and every projects to do some
> introspection ... Time to solve this chicken-and-egg problem.

The service dependency issue is, indeed, a difficult problem to solve.
In the early days of OpenStack, we decided that every service could
assume that a number of base services would be available: a relational
database (MySQL), a message queue (RabbitMQ), and an AuthN/AuthZ token
service (Keystone). That served us well, but we were unable to grow that
set of "base services".

We need more advanced features, like a distributed lock manager
(Zookeeper?), or a secrets vault (Barbican?), but rather than making the
hard decision, we work around their absence in every project, badly
emulating those features using what we have. This has nothing to do with
the big tent or the way we structure projects. It just has to do with
the size of this community. It was easier to agree to depend on MySQL
and RabbitMQ and Keystone when we were 100.

Now, how do we solve it ? First, we need to realize what the issue is,
define language around it. Using the Architecture WG as a vehicle, I
started to push the idea of defining "base services"[1] (resources that
other services can assume will be present). This is the first step:
realizing we do have base services, and need a way to *extend* them.

[1]
https://git.openstack.org/cgit/openstack/arch-wg/tree/proposals/base-services.rst

The next step will be to propose NEW base services. It's simpler than
you think -- the TC will just say it's fine to assume that service X
will be present. We obviously need to pick the right solutions, the ones
that solve the problem set and actually are not horrible to deploy. I
expect the Architecture WG to help in that analysis. But beyond that,
making the decision that it is OK to depend on them is not that hard.

> Stick together, there seems still a chance for future; otherwise, we
> will feel guilty wasting people's life building something that is
> falling apart eventually. Should we kill all "sh**-as-a-service"
> projects and focus on the few "core" services and hope they will meet
> all users' requirements? Or, should we give every project an equal
> chance to be adopted? Who is blocking other services to get adopted?
> How many projects are listed on the project navigator?

I think the focus question is an illusion, as Ed brilliantly explained
in https://blog.leafe.com/openstack-focus/

The issue here is that it's just a lot more profitable career-wise and a
lot less risky to work first-level user-visible features like Machine
Learning as a service, than it is to work on infrastructural services
like Glance, Keystone or Barbican. Developers naturally prefer to go to
shiny objects than to boring technology. As long as their corporate
sponsors are happy with them ignoring critical services, that will
continue. Saying that some of those things are not part of our
community, while they are developed by our community, is sticking our
heads in the sand.

We can certainly influence where those corporate sponsors dedicate their
development resources (and I think we should definitely pursue the base
service stuff, to send a strong signal), but we don't directly control
where the resources are spent.

-- 
Thierry Carrez (ttx)