<div dir="ltr"><div>Hi Steve,</div><div><br></div><div>I am trying to understand the software config implementation. Can you clarify the following:</div><div><br></div><div>i. To use Software config and deploy in a template, instance resource MUST always be accompanied by user_data. User_data should specify how to bootstrap CM tool and signal it. Is that correct?</div>
<div><br></div><div>ii. Supposing we were to use images which do not have cloud-init packaged in them, (and a custom CM tool that won't require bootstrapping on the instance itself), can we still use software config and deploy resources to deploy software on such instances? </div>
<div><br></div><div>iii. If ii. were possible who would signal the deployment resource to indicate that the instance is ready for the deployment? </div><div><br></div><div>Thanks</div><div>Susaant</div></div><div class="gmail_extra">
<br><br><div class="gmail_quote">On Fri, Dec 13, 2013 at 3:46 PM, Steve Baker <span dir="ltr"><<a href="mailto:sbaker@redhat.com" target="_blank">sbaker@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
I've been working on a POC in heat for resources which perform
software configuration, with the aim of implementing this spec
<a href="https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config-spec" target="_blank">https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config-spec</a><br>
<br>
The code to date is here:<br>
<a href="https://review.openstack.org/#/q/topic:bp/hot-software-config,n,z" target="_blank">https://review.openstack.org/#/q/topic:bp/hot-software-config,n,z</a><br>
<br>
What would be helpful now is reviews which give the architectural
approach enough of a blessing to justify fleshing this POC out into
a ready to merge changeset.<br>
<br>
Currently it is possible to:<br>
- create templates containing OS::Heat::SoftwareConfig and
OS::Heat::SoftwareDeployment resources<br>
- deploy configs to OS::Nova::Server, where the deployment resource
remains in an IN_PROGRESS state until it is signalled with the
output values<br>
- write configs which execute shell scripts and report back with
output values that other resources can have access to.<br>
<br>
What follows is an overview of the architecture and implementation
to help with your reviews.<br>
<br>
REST API<br>
========<br>
Like many heat resources, OS::Heat::SoftwareConfig and
OS::Heat::SoftwareDeployment are backed by "real" resources that are
invoked via a REST API. However in this case, the API that is called
is heat itself.<br>
<br>
The REST API for these resources really just act as structured
storage for config and deployments, and the entities are managed via
the REST paths /{tenant_id}/software_configs and
/{tenant_id}/software_deployments:<br>
<a href="https://review.openstack.org/#/c/58878/7/heat/api/openstack/v1/__init__.py" target="_blank">
</a><a href="https://review.openstack.org/#/c/58878/" target="_blank">https://review.openstack.org/#/c/58878/</a><br>
RPC layer of REST API:<br>
<a href="https://review.openstack.org/#/c/58877/" target="_blank">https://review.openstack.org/#/c/58877/</a><br>
DB layer of REST API:<br>
<a href="https://review.openstack.org/#/c/58876" target="_blank">https://review.openstack.org/#/c/58876</a><br>
heatclient lib access to REST API:<br>
<a href="https://review.openstack.org/#/c/58885" target="_blank">https://review.openstack.org/#/c/58885</a><br>
<br>
This data could be stored in a less structured datastore like swift,
but this API has a couple of important implementation details which
I think justify it existing:<br>
- SoftwareConfig resources are immutable once created. There is no
update API to modify an existing config. This gives confidence that
a config can have a long lifecycle without changing, and a certainty
of what exactly is deployed on a server with a given config.<br>
- Fetching all the deployments and configs for a given server is an
operation done repeatedly throughout the lifecycle of the stack, so
is optimized to be able to do in a single operation. This is called
by using the deployments index API call,
/{tenant_id}/software_deployments?server_id=<server_id>. The
resulting list of deployments include the their associated config
data[1].<br>
<br>
OS::Heat::SoftwareConfig resource<br>
=================================<br>
OS::Heat::SoftwareConfig can be used directly in a template, but it
may end be more frequently used in a resource provider template
which provides a resource aimed at a particular configuration
management tool.<br>
<a href="http://docs-draft.openstack.org/79/58879/7/check/gate-heat-docs/911a250/doc/build/html/template_guide/openstack.html#OS::Heat::SoftwareConfig" target="_blank">http://docs-draft.openstack.org/79/58879/7/check/gate-heat-docs/911a250/doc/build/html/template_guide/openstack.html#OS::Heat::SoftwareConfig</a><br>
The contents of the config property will depend on the CM tool being
used, but at least one value in the config map will be the actual
script that the CM tool invokes. An inputs and outputs schema is
also defined here. The group property is used when the deployments
data is actually delivered to the server (more on that later).<br>
<br>
Since a config is immutable, any changes to a
OS::Heat::SoftwareConfig on stack update result in replacement.<br>
<br>
OS::Heat::SoftwareDeployment resource<br>
=====================================<br>
OS::Heat::SoftwareDeployment joins a OS::Heat::SoftwareConfig
resource with a OS::Nova::Server resource. It allows server-specific
input values to be specified that map to the
OS::Heat::SoftwareConfig inputs schema. Output values that are
signaled to the deployment resource are exposed as resource
attributes, using the names specified in the outputs schema. The
OS::Heat::SoftwareDeployment resource remains in an IN_PROGRESS
state until it receives a signal (containing any outputs) from the
server.<br>
<a href="http://docs-draft.openstack.org/79/58879/7/check/gate-heat-docs/911a250/doc/build/html/template_guide/openstack.html#OS::Heat::SoftwareDeployment" target="_blank">http://docs-draft.openstack.org/79/58879/7/check/gate-heat-docs/911a250/doc/build/html/template_guide/openstack.html#OS::Heat::SoftwareDeployment</a><br>
<br>
A deployment has its own actions and statuses that are specific to
what a deployment does, and OS::Heat::SoftwareDeployment maps this
to heat resource statuses and actions:<br>
actions:<br>
DEPLOY -> CREATE<br>
REDEPLOY -> UPDATE<br>
UNDEPLOY -> DELETE<br>
<br>
status (these could use some bikeshedding):<br>
WAITING -> IN_PROGRESS<br>
RECEIVED -> COMPLETE<br>
FAILED -> FAILED<br>
<br>
In the config outputs schema there is a special flag for
error_output. If the signal response contains any value for any of
these error_output outputs then the deployment resource is put into
the FAILED state.<br>
<br>
The SoftwareDeployment class subclasses SignalResponder which means
that a SoftwareDeployment creates an associated user and ec2
keypair. Since the SoftwareDeployment needs to use the resource_id
for the deployment resource uuid, the user_id needs to be stored in
resource-date instead. This non-wip change enables that:<br>
<a href="https://review.openstack.org/#/c/61902/" target="_blank">https://review.openstack.org/#/c/61902/</a><br>
<br>
During create, the deployment REST API is polled until status goes
from WAITING to RECEIVED. When handle_signal is called, the
deployment is updated via the REST API to set the status to RECEIVED
(or FAILED), along with any output values that were received.<br>
<br>
One alarming consequence of having a deployments API is that any
tenant user can create a deployment for any heat-created nova server
and that software will be deployed to that server, which is, um,
powerful.<br>
<br>
There will need to be a deployment policy (probably an
OS::Nova::Server property) which limits to scope of what deployments
are allowed on that server. This could default to deployments in the
same stack, but could still allow deployments from anywhere.<br>
<br>
OS::Nova::Server support<br>
========================<br>
<a href="https://review.openstack.org/#/c/58880" target="_blank">https://review.openstack.org/#/c/58880</a><br>
A new user_data_format=SOFTWARE_CONFIG is currently used to denote
that this server is configured via software config deployments. Like
user_data_format=HEAT_CFNTOOLS, nova_utils.build_userdata is used to
build the cloud-init parts required to support software config.
However like user_data_format=RAW anything specified in user_data
will be parsed as cloud-init data. If user_data is multi-part data
then the parts will be appended to the parts created in
nova_utils.build_userdata.<br>
<br>
The agent used currently is os-collect-config. This is typically
configured to poll for metadata from a particular heat resource via
the CFN API using the configured ec2 keypair. In the current
implementation the resource which is polled is the OS::Nova::Server
itself, since this is the only resource known to exist at server
boot time (deployment resources depend on server resources, so have
not been created yet). The ec2 keypair comes from a user created
implicitly with the server (similar to SignalResponder resources).
This means the template author doesn't need to include
User/AccessKey/AccessPolicy resources in their templates just to
enable os-collect-config metadata polling.<br>
<br>
Until now, polling the metadata for a resource just returns the
metadata which has been stored in the stack resource database. This
implementation changes metadata polling to actually query the
deployments API to return the latest deployments data. This means
deployment state can be stored in one place, and there is no need to
keep various metadata stores updated with any changed state.<br>
<br>
An actual template<br>
==================<br>
<a href="http://paste.openstack.org/show/54988/" target="_blank">http://paste.openstack.org/show/54988/</a><br>
This template contains:<br>
- a config resource<br>
- 2 deployments which deploy that config with 2 different sets of
inputs<br>
- stack outputs which output the results of the deployments<br>
- a server resource<br>
- an os-refresh-config script delivered via cloud-config[2] which
executes config scripts with deployment inputs and signals outputs
to the provided webhook.<br>
<br>
/opt/stack/os-config-refresh/configure.d/55-heat-config-bash is a
hook specific for performing configuration via shell scripts, and
only acts on software config which has group=Heat::Shell. Each
configuration management tool will have its own hook, and will act
on its own group namespace. Each configuration management tool will
also have its own way of passing inputs and outputs. The hooks job
is to invoke the CM tool with the given inputs and script, then
extract the outputs and signal heat.<br>
<br>
The server needs to have the CM tool and the hook already installed,
either by building a golden image or by using cloud-config during
boot.<br>
<br>
Next steps<br>
==========<br>
There is a lot left to do and I'd like to spread the development
load. What happens next entirely depends on feedback to this POC,
but here is my ideal scenario:<br>
- any feedback which causes churn on many of the current changes I
will address<br>
- a volunteer is found to take the REST API/RPC/DB/heatclient
changes and make them ready to merge<br>
- we continue to discuss and refine the resources, the changes to
OS::Nova::Server, and the example shell hook<br>
- volunteers write hooks for different CM tools, Chef and Puppet
hooks will need to be attempted soon to validate this approach.<br>
<br>
Vaguely related changes include:<br>
- Some solution for specifying cloud-init config, either the
intrinsic functions or cloud-init heat resources<br>
- Some heatclient file inclusion mechanism - writing that python
hook in a heat yaml template was a bit painful ;)<br>
<br>
Trying for yourself<br>
===================<br>
- Using diskimage-builder, create an ubuntu image with
tripleo-image-elements os-apply-config, os-refresh-config and
os-collect-config<br>
- Create a local heat branch containing <a href="https://review.openstack.org/#/q/topic:bp/cloud-init-resource,n,z" target="_blank">https://review.openstack.org/#/q/topic:bp/cloud-init-resource,n,z</a>
and <a href="https://review.openstack.org/#/q/topic:bp/hot-software-config,n,z" target="_blank">https://review.openstack.org/#/q/topic:bp/hot-software-config,n,z</a><br>
- launch the above template with your created image<br>
<br>
cheers<br>
<br>
[1]
<a href="https://review.openstack.org/#/c/58877/7/heat/engine/api.py" target="_blank">https://review.openstack.org/#/c/58877/7/heat/engine/api.py</a><br>
[2] This relies on these not-merged intrinsic functions
<a href="https://review.openstack.org/#/q/topic:bp/cloud-init-resource,n,z" target="_blank">https://review.openstack.org/#/q/topic:bp/cloud-init-resource,n,z</a>
</div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div>