[openstack-dev] [tc] [all] TC Report 18-26

Jay Pipes jaypipes at gmail.com
Mon Jul 2 23:13:36 UTC 2018


On 06/27/2018 07:23 PM, Zane Bitter wrote:
> On 27/06/18 07:55, Jay Pipes wrote:
>> Above, I was saying that the scope of the *OpenStack* community is 
>> already too broad (IMHO). An example of projects that have made the 
>> *OpenStack* community too broad are purpose-built telco applications 
>> like Tacker [1] and Service Function Chaining. [2]
>>
>> I've also argued in the past that all distro- or vendor-specific 
>> deployment tools (Fuel, Triple-O, etc [3]) should live outside of 
>> OpenStack because these projects are more products and the relentless 
>> drive of vendor product management (rightfully) pushes the scope of 
>> these applications to gobble up more and more feature space that may 
>> or may not have anything to do with the core OpenStack mission (and 
>> have more to do with those companies' product roadmap).
> 
> I'm still sad that we've never managed to come up with a single way to 
> install OpenStack. The amount of duplicated effort expended on that 
> problem is mind-boggling. At least we tried though. Excluding those 
> projects from the community would have just meant giving up from the 
> beginning.

You have to have motivation from vendors in order to achieve said single 
way of installing OpenStack. I gave up a long time ago on distros and 
vendors to get behind such an effort.

Where vendors see $$$, they will attempt to carve out value 
differentiation. And value differentiation leads to, well, differences, 
naturally.

And, despite what some might misguidedly think, Kubernetes has no single 
installation method. Their *official* setup/install page is here:

https://kubernetes.io/docs/setup/pick-right-solution/

It lists no fewer than *37* (!) different ways of installing Kubernetes, 
and I'm not even including anything listed in the "Custom Solutions" 
section.

> I think Thierry's new map, that collects installer services in a 
> separate bucket (that may eventually come with a separate git namespace) 
> is a helpful way of communicating to users what's happening without 
> forcing those projects outside of the community.

Sure, I agree the separate bucket is useful, particularly when paired 
with information that allows operators to know how stable and/or 
bleeding edge the code is expected to be -- you know, those "tags" that 
the TC spent time curating.

>>> So to answer your question:
>>>
>>> <jaypipes> zaneb: yeah... nobody I know who argues for a small stable 
>>> core (in Nova) has ever said there should be fewer higher layer 
>>> services.
>>> <jaypipes> zaneb: I'm not entirely sure where you got that idea from.
>>
>> Note the emphasis on *Nova* above?
>>
>> Also note that when I've said that *OpenStack* should have a smaller 
>> mission and scope, that doesn't mean that higher-level services aren't 
>> necessary or wanted.
> 
> Thank you for saying this, and could I please ask you to repeat this 
> disclaimer whenever you talk about a smaller scope for OpenStack.

Yes. I shall shout it from the highest mountains. [1]

> Because for those of us working on higher-level services it feels like 
> there has been a non-stop chorus (both inside and outside the project) 
> of people wanting to redefine OpenStack as something that doesn't 
> include us.

I've said in the past (on Twitter, can't find the link right now, but 
it's out there somewhere) something to the effect of "at some point, 
someone just needs to come out and say that OpenStack is, at its core, 
Nova, Neutron, Keystone, Glance and Cinder".

Perhaps this is what you were recollecting. I would use a different 
phrase nowadays to describe what I was thinking with the above.

I would say instead "Nova, Neutron, Cinder, Keystone and Glance [2] are 
a definitive lower level of an OpenStack deployment. They represent a 
set of required integrated services that supply the most basic 
infrastructure for datacenter resource management when deploying OpenStack."

Note the difference in wording. Instead of saying "OpenStack is X", I'm 
saying "These particular services represent a specific layer of an 
OpenStack deployment".

Nowadays, I would further add something to the effect of "Depending on 
the particular use cases and workloads the OpenStack deployer wishes to 
promote, an additional layer of services provides workload orchestration 
and workflow management capabilities. This layer of services include 
Heat, Mistral, Tacker, Service Function Chaining, Murano, etc".

Does that provide you with some closure on this feeling of "non-stop 
chorus" of exclusion that you mentioned above?

> The reason I haven't dropped this discussion is because I really want to 
> know if _all_ of those people were actually talking about something else 
> (e.g. a smaller scope for Nova), or if it's just you. Because you and I 
> are in complete agreement that Nova has grown a lot of obscure 
> capabilities that make it fiendishly difficult to maintain, and that in 
> many cases might never have been requested if we'd had higher-level 
> tools that could meet the same use cases by composing simpler operations.
> 
> IMHO some of the contributing factors to that were:
> 
> * The aforementioned hostility from some quarters to the existence of 
> higher-level projects in OpenStack.
> * The ongoing hostility of operators to deploying any projects outside 
> of Keystone/Nova/Glance/Neutron/Cinder (*still* seen playing out in the 
> Barbican vs. Castellan debate, where we can't even correct one of 
> OpenStack's original sins and bake in a secret store - something k8s 
> managed from day one - because people don't want to install another ReST 
> API even over a backend that they'll already have to install anyway).
> * The illegibility of public Nova interfaces to potential higher-level 
> tools.

I would like to point something else out here. Something that may not be 
pleasant to confront.

Heat's competition (for resources and mindshare) is Kubernetes, plain 
and simple.

Heat's competition is not other OpenStack projects.

Nova's competition is not Kubernetes (despite various people continuing 
to say that it is).

Nova is not an orchestration system. Never was and (as long as I'm 
kicking and screaming) never will be.

Nova's primary competition is:

* Stand-alone Ironic
* oVirt and stand-alone virsh callers
* Parts of VMWare vCenter [3]
* MaaS in some respects
* The *compute provisioning* parts of EC2, Azure, and GCP

This is why there is a Kubernetes OpenStack cloud provider plugin [4].

This plugin uses Nova [5] (which can potentially use Ironic), Cinder, 
Keystone and Neutron to deploy kubelets to act as nodes in a Kubernetes 
cluster and load balancer objects to act as the proxies that k8s itself 
uses when deploying Pods and Services.

Heat's architecture, template language and object constructs are in 
direct competition with Kubernetes' API and architecture, with the 
primary difference being a VM-centric [6] vs. a container-centric object 
model.

Heat's template language is similar to Helm's chart template YAML 
structure [7], and with Heat's evolution to the "convergence model", 
Heat's architecture actually got closer to Kubernetes' architecture: 
that of continually attempting to converge an observed state with a 
desired state.

So, what is Heat to do?

The hype and marketing machine is never-ending, I'm afraid. [8]

I'm not sure there's actually anything that can be done about this. 
Perhaps it is a fait accomplis that Kubernetes/Helm will/has become 
synonymous with "orchestration of things". Perhaps not. I'm not an 
oracle, unfortunately.

Maybe the only thing that Heat can do to fend off the coming doom is to 
make a case that Heat's performance, reliability, feature set or 
integration with OpenStack's other services make it a better candidate 
for orchestrating virtual machine or baremetal workloads on an OpenStack 
deployment than Kubernetes is.

Sorry to be the bearer of bad news,
-jay

[1] I live in Florida, though, which has no mountains. But, when I 
visit, say, North Carolina, I shall certainly shout it from their mountains.

[2] some would also say Castellan, Ironic and Designate belong here.

[3] Though VMWare is still trying to be everything that certain IT 
administrators ever needed, including orchestration, backup services, 
block storage pooling, high availability, quota management, etc etc

[4] 
https://kubernetes.io/docs/concepts/cluster-administration/cloud-providers/#openstack

[5] 
https://github.com/kubernetes/kubernetes/blob/92b81114f43f3ca74988194406957a5d1ffd1c5d/pkg/cloudprovider/providers/openstack/openstack.go#L377

[6] The fact that Heat started as a CloudFormation API clone gave it its 
VM-centricity.

[7] 
https://github.com/kubernetes/helm/blob/master/docs/chart_template_guide/index.md

[8] The Kubernetes' machine has essentially decimated all the other 
"orchestration of things" projects' resources and mindshare, including a 
number of them that were very well architected, well coded, and well 
documented:

* Mesos with Marathon/Aurora
* Rancher
* OpenShift (you know, the original, original one...)
* Nomad
* Docker Swarm/Compose



More information about the OpenStack-dev mailing list