Open Stack

Excerpts from Zane Bitter's message of 2013-04-11 04:38:07 -0700:
> On 11/04/13 02:18, Tripp, Travis S wrote:
> >
> > Zane Bitter wrote:
> >> FWIW it appears to me that CAMP is targeted at the PaaS level,
> >> and while I'm supportive of that effort it doesn't seem to me to
> >> be in scope for Heat, which I see as limited to the IaaS layer.
> >> I'm willing to be re-educated though ;)
> >
> > Perhaps Heat was intended to be at the IaaS level today (?), but there are already platform and application examples in the test templates today (e.g. wordpress). I know from experience that infrastructure has complexity and that things become even more complex when we start considering how applications use the infrastructure.
> 
> The reason for the choice of the Wordpress application in the examples 
> is that you can try it and see a very concrete demonstration that it 
> works. And for some people (not large enterprise IT departments, one 
> suspects) deploying applications that way will be enough. For most large 
> applications, I would imagine something like the Puppet slave template 
> would be more representative, but it doesn't make for a great demo ;)

Right, Heat's examples are showing whole deliverable templates that
deploy a thing. That is useful for showing all of the things Heat can do,
but it violates the principle of separation of concerns.

> >
> >  From my perspective, there are a few fundamental categories of management to consider in the orchestration of applications and infrastructure.
> >
> > + Physical Resource Management  (Machine, Network, Storage, Load Balancing, etc.)
> > + Configuration Management (Setup users, install software, configure software, deploy code)
> > + Performance Management (Availability, Scalability, Placement)
> > + Security Management (Data, Connectivity, Placement)
> >
> > To accomplish all of the above, we have a point of view that cloud orchestration at the enterprise-level cannot be solved with a single engine or tool and that different enterprises will want to leverage their existing investments in their tools. This is why Chef may be used for configuration management (and even infrastructure resource management), but it doesn't make sense to use Chef for monitoring or making policy decisions about when and where to instantiate resources.
> 
> +1 totally agree.

How your app uses the data heat provides to it should remain as a
peripheral concern of Heat. Chef, heat-cfntools, or whatever, can all
be brought in to accomplish things inside an instance. There's a nice
snippet in the heat-cfntools README that explains this:

"""
There are several bootstrap methods for cloudformations:
1. Create image with application ready to go
2. Use cloud-init to run a startup script passed as userdata to the nova
   server create
3. Use the CloudFormation instance helper scripts

   This package contains files required for choice #3.
"""

Because there are many ways to do this, heat-cfntools remains a separate
code tree and an optional thing.

> >
> > If I were to categorize Heat *today* from an external perspective, I might call it an API orchestrator expressed in a mixed declarative / imperative template script language which provides a way to create, update, and delete multiple resources in a coordinated fashion by calling their APIs through resource plugins (including core resources).  The engine doesn't care about what the resource is, it doesn't care about why it is being called, all it knows is that it has been asked to orchestrate a physical resource stack based on an input template.
> 
> That seems basically accurate, although I'm not sure I would describe it 
> as mixed declarative/imperative. I assume you're referring there to the 
> UserData section of a Nova instance (which is basically just a script)?
> 
> Essentially Heat is designed as a declarative interface to the OpenStack 
> APIs. Deploying an interdependent collection of resources - and 
> especially making changes to that collection - by hand or with a custom 
> script can be complex and error-prone, and every user has to do it every 
> time they want to make a change. There's a big advantage in having a 
> description of the old state and the state you want to get to and 
> letting an orchestration engine figure how to get there. Even better if 
> you can keep that description in version control and easily do diffs to 
> see what changed and when.
> 

This is a side tangent, but you actually need to also store the parameters
and diff those too, or you run the risk of making unintended changes. I've
been working out in my head how to write some CLI helpers for automating
things so that developers can encode "this is what I expect to change
when we update the stack" and automation systems can stop the presses
if there are unintended changes.

> What we have _not_ tried to do is anything that is not part of the 
> OpenStack API, except for some temporary resource types for things that 
> should be in OpenStack but weren't at the time - e.g. LBaaS, DBaaS). So 
> the UserData is passed straight through because it's passed straight 
> through in the Nova API as well. It's worth noting that the UserData is 
> extremely constrained in size - in general the point is not to put all 
> of your application in there, it's to put just enough bootstrap to tell 
> the instance to e.g. grab a Chef recipe and go from there, so that you 
> don't have to build it into a custom image.
> 

Right, if you build an image with heat-cfntools installed that runs
cfn-init and cfn-hup, you don't need UserData at all and can just pass
in declarative Metadata to get things done.

We're taking that a step further with os-config-applier in the tripleo
project by just passing in declarative values which the image knows how
to interpert into config file changes. This way the template only encodes
orchestration, and not config file or software management of any kind.

https://github.com/openstack-ops/templates/blob/master/mysql.yaml

Note the hint to OpenStack::ImageBuilder::Elements, which is just a hint
that this template wants an image built with the mysql-migration element.
This is an alternative way to use Heat without cfn-init, and without
having to specify file paths or service names or anything in the template,
those things all remain in the image as part of the software.

> >
> > If I am correct in my categorization of Heat (open for comments!), what this implies is that Heat provides a tightly scoped service for physical resource API orchestration. It doesn't attempt to intermingle knowledge of the resources with the engine, which would distract from its core capability. I like that in theory, it can treat an infrastructure resource (VM) the same as a configuration resource (Chef cookbook) - a set of inputs that will be passed to a plugin at the appropriate time based upon dependency graphing and let the plugin handle it. It doesn't even care if a resource plugin recursively invokes other Heat resources. It is this intended ignorance of what it is doing that makes it work as an API orchestrator.
> 
> I would agree with that assessment.
> >
> > All of which leads me back to the core questions surfacing that I am very interested in:
> >   + Does Heat support alternate ways to express the resource orchestration?
> >   + Does Heat play a greater role in orchestration other than as pre-scripted resource orchestration?
> >
> > And the implied questions are:
> > + Should Heat support a logical way to model stack layers independently and without physical resource binding?
> > + Does this start breaking down into sub services like Nova did (some of which have now been separated)?
> >
> > I believe that as software projects go from small to large, that the modeling and scripting needs change and often become more complex.  Some people will care about cloud portability and some won't. People in different roles will desire fine grained control and others will not. In other words, there is a need for both logical / declarative modeling as well as physical / imperative modeling and they are complementary. I also believe that there can be course grained declarative modeling as well as fine grained declarative modeling.
> >
> > So, ultimately, the question for Heat is how much, how far, and how soon is it going to go in all of these areas?
> 
> These are good questions. I don't really have direct answers, but here 
> is my 2c:
> 
> I agree that long-term we need to have a better story on configuring 
> what runs inside Nova instances. But I'm also wary of scope creep. 
> During the incubation discussions around Heat, there were a lot of folks 
> expressing concern about blessing one particular orchestration engine as 
> an official OpenStack project and, while I disagree with that view when 
> it comes to orchestrating OpenStack APIs, I think it's a very good point 
> when it comes to dealing with non-OpenStack tools.
> 

As I stated above, this line needs to be clear. Things inside the
instances are not at all "Heat". We should provide minimalistic tools,
but if those tools get too deep, we will actually make it harder to
adopt Heat.

> There are multiple, Open Source PaaS solutions that can run on top of 
> OpenStack (e.g. OpenShift Origin, CloudFoundry). I think where OpenStack 
> in general, and Heat in particular, can add the most value is to be the 
> best IaaS base on which to layer the PaaS. (BTW the hosted version of 
> OpenShift currently uses CloudFormation templates.)
> 

I think there is far too much attention given to "is this PaaS or IaaS".

Applications need a platform that gives them access to the infrastructure.
We can see that in the industry where pure PaaS solutions are not nearly
as popular as those that give you the ability to get out to the OS when
you need to.

Heat should orchestrate OpenStack APIs in its core. If TOSCA apps can't
get their jobs done with just the OpenStack APIs, then the pieces they
need should be on the same level as the heat-cfntools.

> That's obviously not the only use case though. I would be in favour of 
> supporting some ways to generate the UserData for an instance in some 
> simpler manner than a shell script. One idea that has been kicked around 
> is Juju integration, and I think that would be a really interesting one 

Zane you kind of just blew my mind a little. Being intimiately familiar
with how Juju works, I'm not sure this makes any sense at all.

Any attempt to make juju charms work on heat is basically reimplementing
the entire juju charm CLI API (relation-{get,set,list}, unit-get,
config-get) to work using Heat's API tools. Juju doesn't just install
things. It sets up juju's own agents to do what Heat already does (spin
up instances, communicate changes between them).

> to look at. Another would be some sort of (pluggable) layer to add 
> native ways of configuring the likes of Puppet and Chef. (Those two 
> ideas are not mutually exclusive, either.)
> 

Puppet and Chef can do this without changing anything in Heat. Just teach
them how to read the Metadata. IIRC, chef already has an ohai plugin to
do just that. Then just have something like:

    Metadata:
      Puppet:
        master-host: xxxxxx
        roles: [x, y, z]