Open Stack

On 25/10/13 09:25 -0700, Clint Byrum wrote:
>Excerpts from Angus Salkeld's message of 2013-10-24 18:48:16 -0700:
>> On 24/10/13 11:54 +0200, Patrick Petit wrote:
>> >Hi Clint,
>> >Thank you! I have few replies/questions in-line.
>> >Cheers
>> >Patrick
>> >On 10/23/13 8:36 PM, Clint Byrum wrote:
>> >>I think this fits into something that I want for optimizing
>> >>os-collect-config as well (our in-instance Heat-aware agent). That is
>> >>a way for us to wait for notification of changes to Metadata without
>> >>polling.
>> >Interesting... If I understand correctly that's kinda replacement of
>> >cfn-hup... Do you have a blueprint pointer or something more
>> >specific? While I see the benefits of it, in-instance notifications
>> >is not really what we are looking for. We are looking for a
>> >notification service that exposes an API whereby listeners can
>> >register for Heat notifications. AWS Alarming / CloudFormation has
>> >that. Why not Ceilometer / Heat? That would be extremely valuable for
>> >those who build PaaS-like solutions above Heat. To say it bluntly,
>> >I'd like to suggest we explore ways to integrate Heat with Marconi.
>>
>> Yeah, I am trying to do a PoC of this now. I'll let you know how
>> it goes.
>>
>> I am trying to implement the following:
>>
>> heat_template_version: 2013-05-23
>> parameters:
>>    key_name:
>>      type: String
>>    flavor:
>>      type: String
>>      default: m1.small
>>    image:
>>      type: String
>>      default: fedora-19-i386-heat-cfntools
>> resources:
>>    config_server:
>>      type: OS::Marconi::QueueServer
>>      properties:
>>        image: {get_param: image}
>>        flavor: {get_param: flavor}
>>        key_name: {get_param: key_name}
>>
>>    configA:
>>      type: OS::Heat::OrderedConfig
>>      properties:
>>        marconi_server: {get_attr: [config_server, url]}
>>        hosted_on: {get_resource: serv1}
>>        script: |
>>          #!/bin/bash
>>          logger "1. hello from marconi"
>>
>>    configB:
>>      type: OS::Heat::OrderedConfig
>>      properties:
>>        marconi_server: {get_attr: [config_server, url]}
>>        hosted_on: {get_resource: serv1}
>>        depends_on: {get_resource: configA}
>>        script: |
>>          #!/bin/bash
>>          logger "2. hello from marconi"
>>
>>    serv1:
>>      type: OS::Nova::Server
>>      properties:
>>        image: {get_param: image}
>>        flavor: {get_param: flavor}
>>        key_name: {get_param: key_name}
>>        user_data: |
>>          #!/bin/sh
>>          # poll <marconi url>/v1/queues/{hostname}/messages
>>          # apply config
>>          # post a response message with any outputs
>>          # delete request message
>>
>
>If I may diverge this a bit, I'd like to consider the impact of
>hosted_on on reusability in templates. hosted_on feels like an
>anti-pattern, and I've never seen anything quite like it. It feels wrong
>for a well contained component to then reach out and push itself onto
>something else which has no mention of it.

Maybe I shouldn't have used hosted_on, it could be role_name/config_queue.

>
>I'll rewrite your template as I envision it working:
>
>resources:
>   config_server:
>     type: OS::Marconi::QueueServer
>     properties:
>       image: {get_param: image}
>       flavor: {get_param: flavor}
>       key_name: {get_param: key_name}
>
>   configA:
>     type: OS::Heat::OrderedConfig
>     properties:
>       marconi_server: {get_attr: [config_server, url]}
>       script: |
>         #!/bin/bash
>         logger "1. hello from marconi"
>
>   configB:
>     type: OS::Heat::OrderedConfig
>     properties:
>       marconi_server: {get_attr: [config_server, url]}
>       depends_on: {get_resource: configA}
>       script: |
>         #!/bin/bash
>         logger "2. hello from marconi"
>
>   serv1:
>     type: OS::Nova::Server
>     properties:
>       image: {get_param: image}
>       flavor: {get_param: flavor}
>       key_name: {get_param: key_name}
>       components:
>         - configA
>         - configB
>       user_data: |
>         #!/bin/sh
>         # poll <marconi url>/v1/queues/{hostname}/messages
>         # apply config
>         # post a response message with any outputs
>         # delete request message
>
>This only becomes obvious why it is important when you want to do this:
>
>    configC:
>      type: OS::Heat::OrderedConfig
>      properties:
>        script: |
>          #!/bin/bash
>          logger "?. I can race with A, no dependency needed"
Well if you put no dependency, it's like any other heat resource
they will be run in parallel (or at least either may go first)
there are lots of configs where this may not be important.
>
>    serv2:
>      type: OS::Nova::Server
>      properties:
>      ...
>      components:
>        - configA
>        - configC
>
>This is proper composition, where the caller defines the components, not
>the callee. Now you can re-use configA with a different component in the
>same template. As we get smarter we can have these configs separate from
>the template and reusable across templates.
>
>Anyway, I'd like to see us stop talking about hosted_on, and if it has
>been implemented, that it be deprecated and eventually removed, as it is
>just plain confusing.

There are pros and cons to both, I don't however think it is helpful
to shutdown conversation of different solutions. Having options that
meet different peoples' needs within the repo is a good thing.

I agree that having config and placement together restrict
reusablity. And this is an important point.

But I still don't like the list of "components" hanging off of the
server. It seems very restrictive to me.

What is awkward is defing tasks as opposed to resources.
So a solution is to define a seperate resource to connect these
things together (like a policy):

::

  config(s)/task(s) 

  task-coordinator-policy-thingy
  {ref: task-coordinator-handle}
  (could be based on a marconi-queue, taskflow, murano)
  1: apply {ref: confA} on {ref: serverX}
  2: run {ref: taskB} on {ref: serverY}

  task-coordinator-handle
  (just to break the dep loop) 

  serverY
  task-coordinator: {ref: fancy-app-handle}


- configs/tasks are independant and reusable
- the coordinator could even have different logic for
   create/update/suspend
- other projects can have an easy route to integrating with Heat
   (they provide a coordinator resource, with lots of cool/complex logic)

-Angus

>
>_______________________________________________
>OpenStack-dev mailing list
>OpenStack-dev at lists.openstack.org
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev