[openstack-dev] [Heat] HOT Software configuration proposal

Patrick Petit patrick.petit at bull.net
Wed Oct 23 17:58:22 UTC 2013


Dear Steve and All,

If I may add up on this already busy thread to share our experience with 
using Heat in large and complex software deployments.

I work on a project which precisely provides additional value at the 
articulation point between resource orchestration automation and 
configuration management. We rely on Heat and chef-solo respectively for 
these base management functions. On top of this, we have developed an 
event-driven workflow to manage the life-cycles of complex software 
stacks which primary purpose is to support middleware components as 
opposed to end-user apps. Our use cases are peculiar in the sense that 
software setup (install, config, contextualization) is not a one-time 
operation issue but a continuous thing that can happen any time in 
life-span of a stack. Users can deploy (and undeploy) apps long time 
after the stack is created. Auto-scaling may also result in an 
asynchronous apps deployment. More about this latter. The framework we 
have designed works well for us. It clearly refers to a PaaS-like 
environment which I understand is not the topic of the HOT software 
configuration proposal(s) and that's absolutely fine with us. However, 
the question for us is whether the separation of software config from 
resources would make our life easier or not. I think the answer is 
definitely yes but at the condition that the DSL extension preserves 
almost everything from the expressiveness of the resource element. In 
practice, I think that a strict separation between resource and 
component will be hard to achieve because we'll always need a little bit 
of application's specific in the resources. Take for example the case of 
the SecurityGroups. The ports open in a SecurityGroup are application 
specific.

Then, designing a Chef or Puppet component type may be harder than it 
looks at first glance. Speaking of our use cases we still need a little 
bit of scripting in the instance's user-data block to setup a working 
chef-solo environment. For example, we run librarian-chef prior to 
starting chef-solo to resolve the cookbook dependencies. A cookbook can 
present itself as a downloadable tarball but it's not always the case. A 
chef component type would have to support getting a cookbook from a 
public or private git repo (maybe subversion), handle situations where 
there is one cookbook per repo or multiple cookbooks per repo, let the 
user choose a particular branch or label, provide ssh keys if it's a 
private repo, and so forth. We support all of this scenarios and so we 
can provide more detailed requirements if needed.

I am not sure adding component relations like the 'depends-on' would 
really help us since it is the job of config management to handle 
software dependencies. Also, it doesn't address the issue of circular 
dependencies. Circular dependencies occur in complex software stack 
deployments. Example. When we setup a Slum virtual cluster, both the 
head node and compute nodes depend on one another to complete their 
configuration and so they would wait for each other indefinitely if we 
were to rely on the 'depends-on'. In addition, I think it's critical to 
distinguish between configuration parameters which are known ahead of 
time, like a db name or user name and password, versus contextualization 
parameters which are known after the fact generally when the instance is 
created. Typically those contextualization parameters are IP addresses 
but not only. The fact packages x,y,z have been properly installed and 
services a,b,c successfully started is contextualization information 
(a.k.a facts) which may be indicative that other components can move on 
to the next setup stage.

The case of complex deployments with or without circular dependencies is 
typically resolved by making the system converge toward the desirable 
end-state through running idempotent recipes. This is our approach. The 
first configuration phase handles parametrization which in general 
brings an instance to CREATE_COMPLETE state. A second phase follows to 
handle contextualization at the stack level. As a matter of fact, a new 
contextualization should be triggered every time an instance enters or 
leave the CREATE_COMPLETE state which may happen any time with 
auto-scaling. In that phase, circular dependencies can be resolved 
because all contextualization data can be compiled globally. Notice that 
Heat doesn't provide a purpose built resource or service like Chef's 
data-bag for the storage and retrieval of metadata. This a gap which IMO 
should be addressed in the proposal. Currently, we use a kludge that is 
to create a fake AWS::AutoScaling::LaunchConfiguration resource to store 
contextualization data in the metadata section of that resource.

Aside from the HOT software configuration proposal(s). There are two 
critical enhancements in Heat that would make software life-cycles 
management much easier. In fact, they are actual blockers for us.

The first one would be to support asynchronous notifications when an 
instance is created or deleted as a result of an auto-scaling decision. 
As stated earlier, contextualization needs to apply in a stack every 
time a instance enters or leaves the CREATE_COMPLETE state. I am not 
referring to a Ceilometer notification but a Heat notification that can 
be consumed by a Heat client.

The second one would be to support a new type of AWS::IAM::User (perhaps 
OS::IAM::User) resource whereby one could pass Keystone credentials to 
be able to specify Ceilometer alarms based on application's specific 
metrics (a.k.a KPIs).

I hope this is making sense to you and can serve as a basis for further 
discussions and refinements.

Cheers,
Patrick

On 10/16/13 12:48 AM, Steve Baker wrote:
> I've just written some proposals to address Heat's HOT software 
> configuration needs, and I'd like to use this thread to get some feedback:
> https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config
> https://wiki.openstack.org/wiki/Heat/Blueprints/native-tools-bootstrap-config
>
> Please read the proposals and reply to the list with any comments or 
> suggestions.
>
> We can spend some time discussing software configuration at tomorrow's 
> Heat meeting, but I fully expect we'll still be in the discussion 
> phase at Hong Kong.
>
> cheers
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


-- 
Patrick Petit
Cloud Computing Principal Architect, Innovative Products
Bull, Architect of an Open World TM
Tél : +33 (0)4 76 29 70 31
Mobile : +33 (0)6 85 22 06 39
http://www.bull.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131023/b4ceb5d3/attachment.html>


More information about the OpenStack-dev mailing list