Open Stack

On 11/04/13 02:18, Tripp, Travis S wrote:
> Hello all,
>
> This has been a great discussion that I would like to chime in on. I am an architect at HP where I just spent two years on a continuous delivery enterprise software product before moving full time to an OpenStack based project 4 months ago.  In the last few months, amongst other things, I have spent a portion of my time working with TOSCA and participating on an interoperability committee. I've also been keeping an eye on CAMP and its concepts.
>
> We are currently trying to better understand how we should be using and viewing Heat's role in the overall ecosystem of application + infrastructure orchestration in a managed enterprise environment. Based on these threads and even Zane's comment, there isn't a real agreement on the level and role that Heat plays, and it seems that this is an ideal time to have this discussion, so +1 for initiating the discussion!

Thanks for kicking this off, I think this is a really important and 
valuable discussion to have.

>
> Zane Bitter wrote:
>> FWIW it appears to me that CAMP is targeted at the PaaS level,
>> and while I'm supportive of that effort it doesn't seem to me to
>> be in scope for Heat, which I see as limited to the IaaS layer.
>> I'm willing to be re-educated though ;)
>
> Perhaps Heat was intended to be at the IaaS level today (?), but there are already platform and application examples in the test templates today (e.g. wordpress). I know from experience that infrastructure has complexity and that things become even more complex when we start considering how applications use the infrastructure.

The reason for the choice of the Wordpress application in the examples 
is that you can try it and see a very concrete demonstration that it 
works. And for some people (not large enterprise IT departments, one 
suspects) deploying applications that way will be enough. For most large 
applications, I would imagine something like the Puppet slave template 
would be more representative, but it doesn't make for a great demo ;)
>
>  From my perspective, there are a few fundamental categories of management to consider in the orchestration of applications and infrastructure.
>
> + Physical Resource Management  (Machine, Network, Storage, Load Balancing, etc.)
> + Configuration Management (Setup users, install software, configure software, deploy code)
> + Performance Management (Availability, Scalability, Placement)
> + Security Management (Data, Connectivity, Placement)
>
> To accomplish all of the above, we have a point of view that cloud orchestration at the enterprise-level cannot be solved with a single engine or tool and that different enterprises will want to leverage their existing investments in their tools. This is why Chef may be used for configuration management (and even infrastructure resource management), but it doesn't make sense to use Chef for monitoring or making policy decisions about when and where to instantiate resources.

+1 totally agree.
>
> If I were to categorize Heat *today* from an external perspective, I might call it an API orchestrator expressed in a mixed declarative / imperative template script language which provides a way to create, update, and delete multiple resources in a coordinated fashion by calling their APIs through resource plugins (including core resources).  The engine doesn't care about what the resource is, it doesn't care about why it is being called, all it knows is that it has been asked to orchestrate a physical resource stack based on an input template.

That seems basically accurate, although I'm not sure I would describe it 
as mixed declarative/imperative. I assume you're referring there to the 
UserData section of a Nova instance (which is basically just a script)?

Essentially Heat is designed as a declarative interface to the OpenStack 
APIs. Deploying an interdependent collection of resources - and 
especially making changes to that collection - by hand or with a custom 
script can be complex and error-prone, and every user has to do it every 
time they want to make a change. There's a big advantage in having a 
description of the old state and the state you want to get to and 
letting an orchestration engine figure how to get there. Even better if 
you can keep that description in version control and easily do diffs to 
see what changed and when.

What we have _not_ tried to do is anything that is not part of the 
OpenStack API, except for some temporary resource types for things that 
should be in OpenStack but weren't at the time - e.g. LBaaS, DBaaS). So 
the UserData is passed straight through because it's passed straight 
through in the Nova API as well. It's worth noting that the UserData is 
extremely constrained in size - in general the point is not to put all 
of your application in there, it's to put just enough bootstrap to tell 
the instance to e.g. grab a Chef recipe and go from there, so that you 
don't have to build it into a custom image.

>
> If I am correct in my categorization of Heat (open for comments!), what this implies is that Heat provides a tightly scoped service for physical resource API orchestration. It doesn't attempt to intermingle knowledge of the resources with the engine, which would distract from its core capability. I like that in theory, it can treat an infrastructure resource (VM) the same as a configuration resource (Chef cookbook) - a set of inputs that will be passed to a plugin at the appropriate time based upon dependency graphing and let the plugin handle it. It doesn't even care if a resource plugin recursively invokes other Heat resources. It is this intended ignorance of what it is doing that makes it work as an API orchestrator.

I would agree with that assessment.
>
> All of which leads me back to the core questions surfacing that I am very interested in:
>   + Does Heat support alternate ways to express the resource orchestration?
>   + Does Heat play a greater role in orchestration other than as pre-scripted resource orchestration?
>
> And the implied questions are:
> + Should Heat support a logical way to model stack layers independently and without physical resource binding?
> + Does this start breaking down into sub services like Nova did (some of which have now been separated)?
>
> I believe that as software projects go from small to large, that the modeling and scripting needs change and often become more complex.  Some people will care about cloud portability and some won't. People in different roles will desire fine grained control and others will not. In other words, there is a need for both logical / declarative modeling as well as physical / imperative modeling and they are complementary. I also believe that there can be course grained declarative modeling as well as fine grained declarative modeling.
>
> So, ultimately, the question for Heat is how much, how far, and how soon is it going to go in all of these areas?

These are good questions. I don't really have direct answers, but here 
is my 2c:

I agree that long-term we need to have a better story on configuring 
what runs inside Nova instances. But I'm also wary of scope creep. 
During the incubation discussions around Heat, there were a lot of folks 
expressing concern about blessing one particular orchestration engine as 
an official OpenStack project and, while I disagree with that view when 
it comes to orchestrating OpenStack APIs, I think it's a very good point 
when it comes to dealing with non-OpenStack tools.

There are multiple, Open Source PaaS solutions that can run on top of 
OpenStack (e.g. OpenShift Origin, CloudFoundry). I think where OpenStack 
in general, and Heat in particular, can add the most value is to be the 
best IaaS base on which to layer the PaaS. (BTW the hosted version of 
OpenShift currently uses CloudFormation templates.)

That's obviously not the only use case though. I would be in favour of 
supporting some ways to generate the UserData for an instance in some 
simpler manner than a shell script. One idea that has been kicked around 
is Juju integration, and I think that would be a really interesting one 
to look at. Another would be some sort of (pluggable) layer to add 
native ways of configuring the likes of Puppet and Chef. (Those two 
ideas are not mutually exclusive, either.)

I hope more folks will chime in with their thoughts here, I think this 
is a very productive discussion to have.

cheers,
Zane.