Open Stack

I'd say it isn't so much cloud-init that you need, but "some kind
of bootstrapper". The point of hot-software-config is to help with
in-instance orchestration. That's not going to happen without some way
to push the desired configuration into the instance.

Excerpts from Susaant Kondapaneni's message of 2014-01-07 11:16:16 -0800:
> We work with images provided by vendors over which we do not always have
> control. So we are considering the cases where vendor image does not come
> installed with cloud-init. Is there a way to support heat software config
> in such scenarios?
> 
> Thanks
> Susaant
> 
> On Mon, Jan 6, 2014 at 4:47 PM, Steve Baker <sbaker at redhat.com> wrote:
> 
> >  On 07/01/14 06:25, Susaant Kondapaneni wrote:
> >
> >  Hi Steve,
> >
> >  I am trying to understand the software config implementation. Can you
> > clarify the following:
> >
> >  i. To use Software config and deploy in a template, instance resource
> > MUST always be accompanied by user_data. User_data should specify how to
> > bootstrap CM tool and signal it. Is that correct?
> >
> >   Yes, currently the user_data contains cfn-init formatted metadata which
> > tells os-collect-config how to poll for config changes. What happens when
> > new config is fetched depends on the os-apply-config templates and
> > os-refresh-config scripts which are already on that image (or set up with
> > cloud-init).
> >
> >  ii. Supposing we were to use images which do not have cloud-init
> > packaged in them, (and a custom CM tool that won't require bootstrapping on
> > the instance itself), can we still use software config and deploy resources
> > to deploy software on such instances?
> >
> >   Currently os-collect-config is more of a requirement than cloud-init,
> > but as Clint said cloud-init does a good job of boot config so you'll need
> > to elaborate on why you don't want to use it.
> >
> >  iii. If ii. were possible who would signal the deployment resource to
> > indicate that the instance is ready for the deployment?
> >
> > os-collect-config polls for the deployment data, and triggers the
> > resulting deployment/config changes. One day this may be performed by a
> > different agent like the unified agent that has been discussed. Currently
> > os-collect-collect polls via a heat-api-cfn metadata call. This too may be
> > done in any number of ways in the future such as messaging or long-polling.
> >
> > So you *could* consume the supplied user_data to know what to poll for
> > subsequent config changes without cloud-init or os-collect-config, but you
> > would have to describe what you're doing in detail for us to know if that
> > sounds like a good idea.
> >
> >
> >
> >  Thanks
> > Susaant
> >
> >
> > On Fri, Dec 13, 2013 at 3:46 PM, Steve Baker <sbaker at redhat.com> wrote:
> >
> >>  I've been working on a POC in heat for resources which perform software
> >> configuration, with the aim of implementing this spec
> >> https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config-spec
> >>
> >> The code to date is here:
> >> https://review.openstack.org/#/q/topic:bp/hot-software-config,n,z
> >>
> >> What would be helpful now is reviews which give the architectural
> >> approach enough of a blessing to justify fleshing this POC out into a ready
> >> to merge changeset.
> >>
> >> Currently it is possible to:
> >> - create templates containing OS::Heat::SoftwareConfig and
> >> OS::Heat::SoftwareDeployment resources
> >> - deploy configs to OS::Nova::Server, where the deployment resource
> >> remains in an IN_PROGRESS state until it is signalled with the output values
> >> - write configs which execute shell scripts and report back with output
> >> values that other resources can have access to.
> >>
> >> What follows is an overview of the architecture and implementation to
> >> help with your reviews.
> >>
> >> REST API
> >> ========
> >> Like many heat resources, OS::Heat::SoftwareConfig and
> >> OS::Heat::SoftwareDeployment are backed by "real" resources that are
> >> invoked via a REST API. However in this case, the API that is called is
> >> heat itself.
> >>
> >> The REST API for these resources really just act as structured storage
> >> for config and deployments, and the entities are managed via the REST paths
> >> /{tenant_id}/software_configs and /{tenant_id}/software_deployments:
> >>
> >> <https://review.openstack.org/#/c/58878/7/heat/api/openstack/v1/__init__.py>
> >> https://review.openstack.org/#/c/58878/
> >> RPC layer of REST API:
> >> https://review.openstack.org/#/c/58877/
> >> DB layer of REST API:
> >> https://review.openstack.org/#/c/58876
> >> heatclient lib access to REST API:
> >> https://review.openstack.org/#/c/58885
> >>
> >> This data could be stored in a less structured datastore like swift, but
> >> this API has a couple of important implementation details which I think
> >> justify it existing:
> >> - SoftwareConfig resources are immutable once created. There is no update
> >> API to modify an existing config. This gives confidence that a config can
> >> have a long lifecycle without changing, and a certainty of what exactly is
> >> deployed on a server with a given config.
> >> - Fetching all the deployments and configs for a given server is an
> >> operation done repeatedly throughout the lifecycle of the stack, so is
> >> optimized to be able to do in a single operation. This is called by using
> >> the deployments index API call,
> >> /{tenant_id}/software_deployments?server_id=<server_id>. The resulting list
> >> of deployments include the their associated config data[1].
> >>
> >> OS::Heat::SoftwareConfig resource
> >> =================================
> >> OS::Heat::SoftwareConfig can be used directly in a template, but it may
> >> end be more frequently used in a resource provider template which provides
> >> a resource aimed at a particular configuration management tool.
> >>
> >> http://docs-draft.openstack.org/79/58879/7/check/gate-heat-docs/911a250/doc/build/html/template_guide/openstack.html#OS::Heat::SoftwareConfig
> >> The contents of the config property will depend on the CM tool being
> >> used, but at least one value in the config map will be the actual script
> >> that the CM tool invokes.  An inputs and outputs schema is also defined
> >> here. The group property is used when the deployments data is actually
> >> delivered to the server (more on that later).
> >>
> >> Since a config is immutable, any changes to a OS::Heat::SoftwareConfig on
> >> stack update result in replacement.
> >>
> >> OS::Heat::SoftwareDeployment resource
> >> =====================================
> >> OS::Heat::SoftwareDeployment joins a OS::Heat::SoftwareConfig resource
> >> with a OS::Nova::Server resource. It allows server-specific input values to
> >> be specified that map to the OS::Heat::SoftwareConfig inputs schema. Output
> >> values that are signaled to the deployment resource are exposed as resource
> >> attributes, using the names specified in the outputs schema. The
> >> OS::Heat::SoftwareDeployment resource remains in an IN_PROGRESS state until
> >> it receives a signal (containing any outputs) from the server.
> >>
> >> http://docs-draft.openstack.org/79/58879/7/check/gate-heat-docs/911a250/doc/build/html/template_guide/openstack.html#OS::Heat::SoftwareDeployment
> >>
> >> A deployment has its own actions and statuses that are specific to what a
> >> deployment does, and OS::Heat::SoftwareDeployment maps this to heat
> >> resource statuses and actions:
> >> actions:
> >> DEPLOY -> CREATE
> >> REDEPLOY -> UPDATE
> >> UNDEPLOY -> DELETE
> >>
> >> status (these could use some bikeshedding):
> >> WAITING -> IN_PROGRESS
> >> RECEIVED -> COMPLETE
> >> FAILED -> FAILED
> >>
> >> In the config outputs schema there is a special flag for error_output. If
> >> the signal response contains any value for any of these error_output
> >> outputs then the deployment resource is put into the FAILED state.
> >>
> >> The SoftwareDeployment class subclasses SignalResponder which means that
> >> a SoftwareDeployment creates an associated user and ec2 keypair. Since the
> >> SoftwareDeployment needs to use the resource_id for the deployment resource
> >> uuid, the user_id needs to be stored in resource-date instead. This non-wip
> >> change enables that:
> >> https://review.openstack.org/#/c/61902/
> >>
> >> During create, the deployment REST API is polled until status goes from
> >> WAITING to RECEIVED. When handle_signal is called, the deployment is
> >> updated via the REST API to set the status to RECEIVED (or FAILED), along
> >> with any output values that were received.
> >>
> >> One alarming consequence of having a deployments API is that any tenant
> >> user can create a deployment for any heat-created nova server and that
> >> software will be deployed to that server, which is, um, powerful.
> >>
> >> There will need to be a deployment policy (probably an OS::Nova::Server
> >> property) which limits to scope of what deployments are allowed on that
> >> server. This could default to deployments in the same stack, but could
> >> still allow deployments from anywhere.
> >>
> >> OS::Nova::Server support
> >> ========================
> >> https://review.openstack.org/#/c/58880
> >> A new user_data_format=SOFTWARE_CONFIG is currently used to denote that
> >> this server is configured via software config deployments. Like
> >> user_data_format=HEAT_CFNTOOLS, nova_utils.build_userdata is used to build
> >> the cloud-init parts required to support software config. However like
> >> user_data_format=RAW anything specified in user_data will be parsed as
> >> cloud-init data. If user_data is multi-part data then the parts will be
> >> appended to the parts created in nova_utils.build_userdata.
> >>
> >> The agent used currently is os-collect-config. This is typically
> >> configured to poll for metadata from a particular heat resource via the CFN
> >> API using the configured ec2 keypair. In the current implementation the
> >> resource which is polled is the OS::Nova::Server itself, since this is the
> >> only resource known to exist at server boot time (deployment resources
> >> depend on server resources, so have not been created yet). The ec2 keypair
> >> comes from a user created implicitly with the server (similar to
> >> SignalResponder resources). This means the template author doesn't need to
> >> include User/AccessKey/AccessPolicy resources in their templates just to
> >> enable os-collect-config metadata polling.
> >>
> >> Until now, polling the metadata for a resource just returns the metadata
> >> which has been stored in the stack resource database. This implementation
> >> changes metadata polling to actually query the deployments API to return
> >> the latest deployments data. This means deployment state can be stored in
> >> one place, and there is no need to keep various metadata stores updated
> >> with any changed state.
> >>
> >> An actual template
> >> ==================
> >> http://paste.openstack.org/show/54988/
> >> This template contains:
> >> - a config resource
> >> - 2 deployments which deploy that config with 2 different sets of inputs
> >> - stack outputs which output the results of the deployments
> >> - a server resource
> >> - an os-refresh-config script delivered via cloud-config[2] which
> >> executes config scripts with deployment inputs and signals outputs to the
> >> provided webhook.
> >>
> >> /opt/stack/os-config-refresh/configure.d/55-heat-config-bash is a hook
> >> specific for performing configuration via shell scripts, and only acts on
> >> software config which has group=Heat::Shell. Each configuration management
> >> tool will have its own hook, and will act on its own group namespace. Each
> >> configuration management tool will also have its own way of passing inputs
> >> and outputs. The hooks job is to invoke the CM tool with the given inputs
> >> and script, then extract the outputs and signal heat.
> >>
> >> The server needs to have the CM tool and the hook already installed,
> >> either by building a golden image or by using cloud-config during boot.
> >>
> >> Next steps
> >> ==========
> >> There is a lot left to do and I'd like to spread the development load.
> >> What happens next entirely depends on feedback to this POC, but here is my
> >> ideal scenario:
> >> - any feedback which causes churn on many of the current changes I will
> >> address
> >> - a volunteer is found to take the REST API/RPC/DB/heatclient changes and
> >> make them ready to merge
> >> - we continue to discuss and refine the resources, the changes to
> >> OS::Nova::Server, and the example shell hook
> >> - volunteers write hooks for different CM tools, Chef and Puppet hooks
> >> will need to be attempted soon to validate this approach.
> >>
> >> Vaguely related changes include:
> >> - Some solution for specifying cloud-init config, either the intrinsic
> >> functions or cloud-init heat resources
> >> - Some heatclient file inclusion mechanism - writing that python hook in
> >> a heat yaml template was a bit painful ;)
> >>
> >> Trying for yourself
> >> ===================
> >> - Using diskimage-builder, create an ubuntu image with
> >> tripleo-image-elements os-apply-config, os-refresh-config and
> >> os-collect-config
> >> - Create a local heat branch containing
> >> https://review.openstack.org/#/q/topic:bp/cloud-init-resource,n,z and
> >> https://review.openstack.org/#/q/topic:bp/hot-software-config,n,z
> >> - launch the above template with your created image
> >>
> >> cheers
> >>
> >> [1] https://review.openstack.org/#/c/58877/7/heat/engine/api.py
> >> [2] This relies on these not-merged intrinsic functions
> >> https://review.openstack.org/#/q/topic:bp/cloud-init-resource,n,z
> >>
> >> _______________________________________________
> >> OpenStack-dev mailing list
> >> OpenStack-dev at lists.openstack.org
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >>
> >
> >
> > _______________________________________________
> > OpenStack-dev mailing listOpenStack-dev at lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >
> >
> > _______________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >