Open Stack

On 07/01/14 06:25, Susaant Kondapaneni wrote:
> Hi Steve,
>
> I am trying to understand the software config implementation. Can you
> clarify the following:
>
> i. To use Software config and deploy in a template, instance resource
> MUST always be accompanied by user_data. User_data should specify how
> to bootstrap CM tool and signal it. Is that correct?
>
Yes, currently the user_data contains cfn-init formatted metadata which
tells os-collect-config how to poll for config changes. What happens
when new config is fetched depends on the os-apply-config templates and
os-refresh-config scripts which are already on that image (or set up
with cloud-init).
> ii. Supposing we were to use images which do not have cloud-init
> packaged in them, (and a custom CM tool that won't require
> bootstrapping on the instance itself), can we still use software
> config and deploy resources to deploy software on such instances? 
>
Currently os-collect-config is more of a requirement than cloud-init,
but as Clint said cloud-init does a good job of boot config so you'll
need to elaborate on why you don't want to use it.
> iii. If ii. were possible who would signal the deployment resource to
> indicate that the instance is ready for the deployment?
os-collect-config polls for the deployment data, and triggers the
resulting deployment/config changes. One day this may be performed by a
different agent like the unified agent that has been discussed.
Currently os-collect-collect polls via a heat-api-cfn metadata call.
This too may be done in any number of ways in the future such as
messaging or long-polling.

So you *could* consume the supplied user_data to know what to poll for
subsequent config changes without cloud-init or os-collect-config, but
you would have to describe what you're doing in detail for us to know if
that sounds like a good idea.

>
> Thanks
> Susaant
>
>
> On Fri, Dec 13, 2013 at 3:46 PM, Steve Baker <sbaker at redhat.com
> <mailto:sbaker at redhat.com>> wrote:
>
>     I've been working on a POC in heat for resources which perform
>     software configuration, with the aim of implementing this spec
>     https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config-spec
>
>     The code to date is here:
>     https://review.openstack.org/#/q/topic:bp/hot-software-config,n,z
>
>     What would be helpful now is reviews which give the architectural
>     approach enough of a blessing to justify fleshing this POC out
>     into a ready to merge changeset.
>
>     Currently it is possible to:
>     - create templates containing OS::Heat::SoftwareConfig and
>     OS::Heat::SoftwareDeployment resources
>     - deploy configs to OS::Nova::Server, where the deployment
>     resource remains in an IN_PROGRESS state until it is signalled
>     with the output values
>     - write configs which execute shell scripts and report back with
>     output values that other resources can have access to.
>
>     What follows is an overview of the architecture and implementation
>     to help with your reviews.
>
>     REST API
>     ========
>     Like many heat resources, OS::Heat::SoftwareConfig and
>     OS::Heat::SoftwareDeployment are backed by "real" resources that
>     are invoked via a REST API. However in this case, the API that is
>     called is heat itself.
>
>     The REST API for these resources really just act as structured
>     storage for config and deployments, and the entities are managed
>     via the REST paths /{tenant_id}/software_configs and
>     /{tenant_id}/software_deployments:
>     <https://review.openstack.org/#/c/58878/7/heat/api/openstack/v1/__init__.py>https://review.openstack.org/#/c/58878/
>     RPC layer of REST API:
>     https://review.openstack.org/#/c/58877/
>     DB layer of REST API:
>     https://review.openstack.org/#/c/58876
>     heatclient lib access to REST API:
>     https://review.openstack.org/#/c/58885
>
>     This data could be stored in a less structured datastore like
>     swift, but this API has a couple of important implementation
>     details which I think justify it existing:
>     - SoftwareConfig resources are immutable once created. There is no
>     update API to modify an existing config. This gives confidence
>     that a config can have a long lifecycle without changing, and a
>     certainty of what exactly is deployed on a server with a given config.
>     - Fetching all the deployments and configs for a given server is
>     an operation done repeatedly throughout the lifecycle of the
>     stack, so is optimized to be able to do in a single operation.
>     This is called by using the deployments index API call,
>     /{tenant_id}/software_deployments?server_id=<server_id>. The
>     resulting list of deployments include the their associated config
>     data[1].
>
>     OS::Heat::SoftwareConfig resource
>     =================================
>     OS::Heat::SoftwareConfig can be used directly in a template, but
>     it may end be more frequently used in a resource provider template
>     which provides a resource aimed at a particular configuration
>     management tool.
>     http://docs-draft.openstack.org/79/58879/7/check/gate-heat-docs/911a250/doc/build/html/template_guide/openstack.html#OS::Heat::SoftwareConfig
>     The contents of the config property will depend on the CM tool
>     being used, but at least one value in the config map will be the
>     actual script that the CM tool invokes.  An inputs and outputs
>     schema is also defined here. The group property is used when the
>     deployments data is actually delivered to the server (more on that
>     later).
>
>     Since a config is immutable, any changes to a
>     OS::Heat::SoftwareConfig on stack update result in replacement.
>
>     OS::Heat::SoftwareDeployment resource
>     =====================================
>     OS::Heat::SoftwareDeployment joins a OS::Heat::SoftwareConfig
>     resource with a OS::Nova::Server resource. It allows
>     server-specific input values to be specified that map to the
>     OS::Heat::SoftwareConfig inputs schema. Output values that are
>     signaled to the deployment resource are exposed as resource
>     attributes, using the names specified in the outputs schema. The
>     OS::Heat::SoftwareDeployment resource remains in an IN_PROGRESS
>     state until it receives a signal (containing any outputs) from the
>     server.
>     http://docs-draft.openstack.org/79/58879/7/check/gate-heat-docs/911a250/doc/build/html/template_guide/openstack.html#OS::Heat::SoftwareDeployment
>
>     A deployment has its own actions and statuses that are specific to
>     what a deployment does, and OS::Heat::SoftwareDeployment maps this
>     to heat resource statuses and actions:
>     actions:
>     DEPLOY -> CREATE
>     REDEPLOY -> UPDATE
>     UNDEPLOY -> DELETE
>
>     status (these could use some bikeshedding):
>     WAITING -> IN_PROGRESS
>     RECEIVED -> COMPLETE
>     FAILED -> FAILED
>
>     In the config outputs schema there is a special flag for
>     error_output. If the signal response contains any value for any of
>     these error_output outputs then the deployment resource is put
>     into the FAILED state.
>
>     The SoftwareDeployment class subclasses SignalResponder which
>     means that a SoftwareDeployment creates an associated user and ec2
>     keypair. Since the SoftwareDeployment needs to use the resource_id
>     for the deployment resource uuid, the user_id needs to be stored
>     in resource-date instead. This non-wip change enables that:
>     https://review.openstack.org/#/c/61902/
>
>     During create, the deployment REST API is polled until status goes
>     from WAITING to RECEIVED. When handle_signal is called, the
>     deployment is updated via the REST API to set the status to
>     RECEIVED (or FAILED), along with any output values that were received.
>
>     One alarming consequence of having a deployments API is that any
>     tenant user can create a deployment for any heat-created nova
>     server and that software will be deployed to that server, which
>     is, um, powerful.
>
>     There will need to be a deployment policy (probably an
>     OS::Nova::Server property) which limits to scope of what
>     deployments are allowed on that server. This could default to
>     deployments in the same stack, but could still allow deployments
>     from anywhere.
>
>     OS::Nova::Server support
>     ========================
>     https://review.openstack.org/#/c/58880
>     A new user_data_format=SOFTWARE_CONFIG is currently used to denote
>     that this server is configured via software config deployments.
>     Like user_data_format=HEAT_CFNTOOLS, nova_utils.build_userdata is
>     used to build the cloud-init parts required to support software
>     config. However like user_data_format=RAW anything specified in
>     user_data will be parsed as cloud-init data. If user_data is
>     multi-part data then the parts will be appended to the parts
>     created in nova_utils.build_userdata.
>
>     The agent used currently is os-collect-config. This is typically
>     configured to poll for metadata from a particular heat resource
>     via the CFN API using the configured ec2 keypair. In the current
>     implementation the resource which is polled is the
>     OS::Nova::Server itself, since this is the only resource known to
>     exist at server boot time (deployment resources depend on server
>     resources, so have not been created yet). The ec2 keypair comes
>     from a user created implicitly with the server (similar to
>     SignalResponder resources). This means the template author doesn't
>     need to include User/AccessKey/AccessPolicy resources in their
>     templates just to enable os-collect-config metadata polling.
>
>     Until now, polling the metadata for a resource just returns the
>     metadata which has been stored in the stack resource database.
>     This implementation changes metadata polling to actually query the
>     deployments API to return the latest deployments data. This means
>     deployment state can be stored in one place, and there is no need
>     to keep various metadata stores updated with any changed state.
>
>     An actual template
>     ==================
>     http://paste.openstack.org/show/54988/
>     This template contains:
>     - a config resource
>     - 2 deployments which deploy that config with 2 different sets of
>     inputs
>     - stack outputs which output the results of the deployments
>     - a server resource
>     - an os-refresh-config script delivered via cloud-config[2] which
>     executes config scripts with deployment inputs and signals outputs
>     to the provided webhook.
>
>     /opt/stack/os-config-refresh/configure.d/55-heat-config-bash is a
>     hook specific for performing configuration via shell scripts, and
>     only acts on software config which has group=Heat::Shell. Each
>     configuration management tool will have its own hook, and will act
>     on its own group namespace. Each configuration management tool
>     will also have its own way of passing inputs and outputs. The
>     hooks job is to invoke the CM tool with the given inputs and
>     script, then extract the outputs and signal heat.
>
>     The server needs to have the CM tool and the hook already
>     installed, either by building a golden image or by using
>     cloud-config during boot.
>
>     Next steps
>     ==========
>     There is a lot left to do and I'd like to spread the development
>     load. What happens next entirely depends on feedback to this POC,
>     but here is my ideal scenario:
>     - any feedback which causes churn on many of the current changes I
>     will address
>     - a volunteer is found to take the REST API/RPC/DB/heatclient
>     changes and make them ready to merge
>     - we continue to discuss and refine the resources, the changes to
>     OS::Nova::Server, and the example shell hook
>     - volunteers write hooks for different CM tools, Chef and Puppet
>     hooks will need to be attempted soon to validate this approach.
>
>     Vaguely related changes include:
>     - Some solution for specifying cloud-init config, either the
>     intrinsic functions or cloud-init heat resources
>     - Some heatclient file inclusion mechanism - writing that python
>     hook in a heat yaml template was a bit painful ;)
>
>     Trying for yourself
>     ===================
>     - Using diskimage-builder, create an ubuntu image with
>     tripleo-image-elements os-apply-config, os-refresh-config and
>     os-collect-config
>     - Create a local heat branch containing
>     https://review.openstack.org/#/q/topic:bp/cloud-init-resource,n,z
>     and https://review.openstack.org/#/q/topic:bp/hot-software-config,n,z
>     - launch the above template with your created image
>
>     cheers
>
>     [1] https://review.openstack.org/#/c/58877/7/heat/engine/api.py
>     [2] This relies on these not-merged intrinsic functions
>     https://review.openstack.org/#/q/topic:bp/cloud-init-resource,n,z
>
>     _______________________________________________
>     OpenStack-dev mailing list
>     OpenStack-dev at lists.openstack.org
>     <mailto:OpenStack-dev at lists.openstack.org>
>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140107/1136483c/attachment.html>