[openstack-dev] [Heat] HOT software configuration refined after design summit discussions

Steve Baker sbaker at redhat.com
Tue Nov 19 20:40:54 UTC 2013

On 11/19/2013 08:37 PM, Thomas Spatzier wrote:
> Steve Baker <sbaker at redhat.com> wrote on 18.11.2013 21:52:04:
>> From: Steve Baker <sbaker at redhat.com>
>> To: openstack-dev at lists.openstack.org,
>> Date: 18.11.2013 21:54
>> Subject: Re: [openstack-dev] [Heat] HOT software configuration
>> refined after design summit discussions
>> On 11/19/2013 02:22 AM, Thomas Spatzier wrote:
>>> Hi all,
>>> I have reworked the wiki page [1] I created last week to reflect
>>> discussions we had on the mail list and in IRC. From ML discussions
> last
>>> week it looked like we were all basically on the same page (with some
>>> details to be worked out), and I hope the new draft eliminates some
>>> confusion that the original draft had.
>>> [1]
> https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config-WIP
>> Thanks Thomas, this looks really good. I've actually started on a POC
>> which maps to this model.
> Good to hear that, Steve :-)
> Now that we are converging, should we consolidate the various wiki pages
> and just have one? E.g. copy the complete contents of
> hot-software-config-WIP to your original hot-software-config, or deprecate
> all others and make hot-software-config-WIP the master?
Lets just bless hot-software-config-WIP and add to it as we flesh out
the implementation.

>> I've used different semantics which you may actually prefer some of,
>> please comment below.
>> Resource types:
>> SoftwareConfig -> SoftwareConfig (yay!)
>> SoftwareDeployment -> SoftwareApplier - less typing, less mouth-fatigue
> I'm ok with SoftwareApplier. If we don't hear objections, I can change it
> in the wiki.
>> SoftwareConfig properties:
>> parameters -> inputs - just because parameters is overloaded already.
> Makes sense.
>> Although if the CM tool has their own semantics for inputs then that
>> should be used in that SoftwareConfig resource implementation instead.
>> outputs -> outputs
>> SoftwareApplier properties:
>> software_config -> apply_config - because there will sometimes be a
>> corresponding remove_config
> Makes sense, and the remove_config thought is a very good point!
>> server -> server
>> parameters -> input_values - to match the 'inputs' schema property in
>> SoftwareConfig
> Agree on input_values.
>> Other comments on hot-software-config-WIP:
>> Regarding apply_config/remove_config, if a SoftwareApplier resource is
>> deleted it should trigger any remove_config and wait for the server to
>> acknowledge when that is complete. This allows for any
>> evacuation/deregistering workloads to be executed.
>> I'm unclear yet what the SoftwareConfig 'role' is for, unless the role
>> specifies the contract for a given inputs and outputs schema? How would
>> this be documented or enforced? I'm inclined to leave it out for now.
> So about 'role', as I stated in the wiki, my thinking was that there will
> be different SoftwareConfig and SoftwareApplier implementations per CM tool
> (more on that below), since all CM tools will probably have their specific
> metadata and runtime implementation. So in my example I was using Chef, and
> 'role' is just a Chef concept, i.e. you take a cookbook and configure a
> specific Chef role on a server.
OK, its Chef specific; I'm fine with that.
>> It should be possible to write a SoftwareConfig type for a new CM tool
>> as a provider template. This has some nice implications for deployers
>> and users.
> I think provider templates are a good thing to have clean componentization
> for re-use. However, I think it still would be good to allow users to
> define their SoftwareConfigs inline in a template for simple use cases. I
> heard that requirement in several posts on the ML last week.
> The question is whether we can live with a single implementation of
> SoftwareConfig and SoftwareApplier then (see also below).
Yes, a provider template would encapsulate some base SoftwareConfig
resource type, but users would be free to use this type inline in their
template too.
>> My hope is that there will not need to be a different SoftwareApplier
>> type written for each CM tool. But maybe there will be one for each
>> delivery mechanism. The first implementation will use metadata polling
>> and signals, another might use Marconi. Bootstrapping an image to
>> consume a given CM tool and applied configuration data is something that
>> we need to do, but we can make it beyond the scope of this particular
>> proposal.
> I was thinking about a single implementation, too. However, I cannot really
> imagine how one single implementation could handle both the different
> metadata of different CM tools, and different runtime implementation. I
> think we would want to support at least a handful of the most favorite
> tools, but cannot see at the moment how to cover them all in one
> implementation. My thought was that there could be a super-class for common
> behavior, and then plugins with specific behavior for each tool.
> Anyway, all of that needs to be verified, so working on PoC patches is
> definitely the right thing to do. For example, if we work on implementation
> for two CM tools (e.g. Chef and simple scripts), we can probably see if one
> common implementation is possible or not.
> Someone from our team is going to write a provider for Chef to try things
> out. I think that can be aligned nicely with your work.
I think there needs to a CM tool specific agent delivered to the server
which os-collect-config invokes. This agent will transform the config
data (input values, CM script, CM specific specialness) to a CM tool

How to define and deliver this agent is the challenge. Some options are:
1) install it as part of the image customization/bootstrapping (golden
images or cloud-init)
2) define a (mustache?) template in the SoftwareConfig which
os-collect-config transforms into the agent script, which
os-collect-config then executes
3) a CM tool specific implementation of SoftwareApplier builds and
delivers a complete agent to os-collect-config which executes it

I may be leaning towards 3) at the moment. Hopefully any agent can be
generated with a sufficiently sophisticated base SoftwareApplier type,
plus maybe some richer intrinsic functions.
>> The POC I'm working on is actually backed by a REST API which does dumb
>> (but structured) storage of SoftwareConfig and SoftwareApplier entities.
>> This has some interesting implications for managing SoftwareConfig
>> resources outside the context of the stack which uses them, but lets not
>> worry too much about that *yet*.
> Sounds good. We are also defining some blueprints to break down the overall
> software config topic. We plan to share them later this week, and then we
> can consolidate with your plans and see how we can best join forces.
At this point it would be very helpful to spec out how specific CM tools
are invoked with given inputs, script, and CM tool specific options.

Maybe if you start with shell scripts, cfn-init and chef then we can all
contribute other CM tools like os-config-applier, puppet, ansible,

Hopefully by then my POC will at least be able to create resources, if
not deliver some data to servers.

More information about the OpenStack-dev mailing list