Open Stack

On 28/10/13 16:26 -0700, Clint Byrum wrote:

[snip]

>> >
>> My components proposals had no hosted_on, but I've been thinking about
>> the implications of implementing software configuration as resources,
>> and one of the natural consequences might be that hosted_on is the best
>> way of establishing the relationship with compute and its
>> configurations. Let me elaborate.
>>
>> Lets say that Heat has resource types for software configuration, with
>> the following behaviours:
>> * like other resources, a config resource goes into CREATE IN_PROGRESS
>> as soon as its dependencies are satisfied (these dependencies may be
>> values signalled from other config resources)
>> * a config resource goes to state CREATE COMPLETE when it receives a
>> signal that configuration on the compute resource is complete (by some
>> mechanism; wait condition, marconi message, whatevs)
>> * the relationship between config resources and compute resources are
>> achieved with existing intrinsic functions (get_resource, get_attr)
>>
>> This lifecycle behaviour means that a configuration resource can only
>> run on a single compute resource, and that relationship needs to be
>> established somehow. Config resources will have a quirk in that they
>> need to provide 2 sources of configuration data at different times:
>> 1) cloud-init config-data (or other boot-only mechanism), which must be
>> available when the compute resource goes into CREATE IN_PROGRESS
>> 2) the actual configuration data (oac metadata, puppet manifest, chef
>> recipe) which the compute resource needs to be able to fetch and execute
>> whenever it becomes available.
>>
>> The data in 1) implies that the compute needs to depend on the config,
>> but then all concurrency is lost (this won't matter for a
>> cloud-init-only config resource).  Either way, the data for 1) will need
>> to be available when the config resource is still in state INIT
>> COMPLETE, which may impose limitations on how that is defined (ie
>> get_resource, get_attr not allowed).
>>
>> So, 2 concrete examples for handling config/compute dependencies:
>>
>> hosted_on
>> ---------
>> resources:
>>   configA
>>     type: Heat::ScriptConfig
>>     properties:
>>        hosted_on:
>>          get_resource: the_server
>>        script: |
>>          #!/bin/bash
>>          logger "1. hello config A"
>>   configB
>>     type: Heat::ScriptConfig
>>     properties:
>>        hosted_on:
>>          get_resource: the_server
>>        script: |
>>          #!/bin/bash
>>          logger "1. hello config B"
>>   the_server:
>>     type: OS::Nova::Server
>>
>> Here, configA and configB go into CREATE IN_PROGRESS as soon as
>> the_server is CREATE COMPLETE. configA and configB go into CREATE
>> COMPLETE when heat-engine receives a signal that they are complete. This
>> signal may include values that other resources depend on to start their
>> creation.
>>
>
>My biggest objection to hosted_on is that if I want to reuse configA.. I
>can't without an additional level of abstraction. This feels awkward,
>and will be an awkward thing for users. In addition, it is very unclear
>what what will actually be on "the_server" when looking at "the_server".
>
>Nobody has answered my question where else this sort of pattern is used.
>
>> OS::Nova::Server config_resources
>> ---------------------------------
>> resources:
>>   configA
>>     type: Heat::ScriptConfig
>>     properties:
>>        script: |
>>          #!/bin/bash
>>          logger "1. hello config A"
>>   configB
>>     type: Heat::ScriptConfig
>>     properties:
>>        script: |
>>          #!/bin/bash
>>          logger "1. hello config B"
>>   the_server:
>>     type: OS::Nova::Server
>>     properties:
>>       config_resources:
>>         - {get_resource: configA}
>>         - {get_resource: configB}
>>
>> Here there would need to be some bypassing of dependency calculation to
>> allow configA and configB to be created after the_server, maybe by:
>> * special treatment of config_resources to prevent dependencies being
>> created
>> * a get_resource variant which doesn't create a hard dependency
>> (get_resource_deferred?)
>>
>
>IMO no, these are not best expressed as resources in the same way that
>python classes are not expressed as objects.
>
>The template above reads more like code. If used as a component, configA
>remains reusable throughout the rest of the template as it grows and
>changes, and it is very obvious what is expected on "the_server". Each
>server using configA gets its very own completion waitcondition for that
>instantiation of the component.
>
>This is why I liked the components concept, rather than have them as
>resources. It seems clear to me that components are parse-time objects,
>so if I refer to the same component 4 times, I really want 4 copies of
>it. Resources, on the other hand, are real things that are instantiated.
>
>> Neither the hosted_on nor the config_resources behaviours are ideal, but
>> I'm leaning towards hosted_on at the moment since it doesn't require any
>> new soft-dependency mechanism.
>>
>
>IMO hosted_on is a new dependency mechanism, so avoiding the more clear
>paradigm for the reason of not introducing dependency mechanism seems a
>bit backwards too.
>
>> As for composability, what *actually* needs to be composable is the
>> contents of the script (manifest, recipe) property. Everything else in a
>> config resource is just stack-specific wiring.  There are a couple of
>> ways this composability could be achieved:
>> 1) resource provider in the user environment which specifies the script
>> property
>> 2) __include__ or some equivalent client-side inclusion mechanism
>>
>
>At first it seems like all of it is just per-stack wiring, but the point
>is that even just inside a stack, you will want to reuse whole components,
>not just the script/cloud-config/etc.
>
>I've done a client side __include__ method for TripleO. It works fine
>actually. But since we're developing a language, we need to think
>longer and harder than "works for me", otherwise we will make the PHP
>of orchestration languages.

Zane had an idea for a "FileRef" of some kind that I nealy had time
to implement. This would work is a simerlar way to the way template
resources are uploaded into the "files" structure.

>
>I'd like to see components as a first class facility in HOT so that users
>can compose templates in a logical straight forward manner. The proposals
>that you have laid out provide workable but less discoverable mechanisms.

At the moment we have something that alters a server in a somewhat
simlar way to configuring - volume attaching.

resources:
   server:
     bla...
   volume:
     bla...
   volume_attachment:
     volume_id: {Ref: volume}
     server_id: {Ref: server}


We could model config install in a simerlar way

resources:
   server:
     bla...
   swift_file:
     bla...
   config_installer:
     config_url: {Ref: swift_file}
     server_id: {Ref: server}

This could be shortened to a url:

resources:
   server:
     bla...
   config_installer:
     server_id: {Ref: server}
     config_content: 
       str_replace: 
         template: {get_file: config_url}
         params:
           foo: whatever

- you could reuse the config in multiple places
- the installer could pass variables into config
   (and potentially get outputs out for dynamic attributes)


[I hope we get big whiteboards at summit :-) ]

-Angus

>_______________________________________________
>OpenStack-dev mailing list
>OpenStack-dev at lists.openstack.org
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev