Open Stack

On 10/17/2013 03:11 AM, Thomas Spatzier wrote:
> Hi Steve,
>
> thanks a lot taking the effort to write all this down. I had a look at both
> wiki pages and have some comments below. This is really from the top of my
> head, and I guess I have to spend some more time thinking about it, but I
> wanted to provide some feedback anyway.
>
> On "components vs. resources":
> So the proposal says clearly that the resource concept is only used for
> things that get accessed and managed via their APIs, i.e. services provided
> by something external to Heat (nova, cinder, etc), while software is
> different and therefore modeled as components, which is basically fine (and
> I also suggested this in my initial proposal .. but was never quite sure).
> Anyway, I think we also need some APIs to access software components (not
> the actual installed software, but the provider managing it), so we can get
> the state of a component, and probably also manage the state to do
> meaningful orchestration. That would bring it close to the resource concept
> again, or components (the providers) would have to provide some means for
> getting access to state etc.
As I've mentioned, messaging/sync is out of scope for these blueprints
but it would be good to start exploring the options now.
> Why no processing of intrinsic functions in config block?
> ... that was actually a question that came up when I read this first, but
> maybe is resolved by some text further down in the wiki. But I wanted to
> ask for clarification. I thought having intrinsic function could be helpful
> for passing parameters around, and also implying dependencies. A bit
> further down some concept for parameter passing to the config providers is
> introduced, and for filling the parameters, intrinsic functions can be
> used. So do I get it right that this would enable the dependency building
> and data passing?
Some reasons for not allowing intrinsic functions in components:
- it allows component config to be purely represented in the syntax of
the CM tool without any reference to Heat constructs
- it allows the CM's native variable handling to be used instead of
bypassing it by doing config substitutions
- it simplifies the implementation of components in heat engine - no
resource or component dependencies, no function evaluation
> Regarding pointer from a server's components section to components vs. a
> hosted_on relationship:
> The current proposal is in fact (or probably) isomorphic to the hosted_on
> links from my earlier proposal. However, having pointers from the lower
> layer (servers) to the upper layers (software) seems a bit odd to me. It
> would be really nice to get clean decoupling of software and infrastructure
> and not just the ability to copy and paste the components and then having
> to define server resources specifically to point to components. The
> ultimate goal would be to have app layer models and infrastructure models
> (e.g. using the environments concept and provider resources) and some way
> of binding app components to one or multiple servers per deployment (single
> server in test, clustered in production).
> Maybe some layer in between is necessary, because neither my earlier
> "hosted_on" proposal nor the current proposal does that.
https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config#Composability

To me, this addresses the separation of architecture from application
configuration. Components don't make any reference to the architecture
of the stack. Server resources only refer to components by name. I would
say that specifying what pieces of configuration run where is in the
domain of architecture. You could even choose component names which make
no reference to the specific application - making the OS::Nova::Server
resource definitions a more pure representation of architecture.
> Why no depends_on (or just dependency) between components?
> Ordering in components is ok, but I think it should be possible to express
> dependencies between components across servers. Whether or not a
> "depends_on" relationship is the right things to express this, or just a
> more simple dependency notation can be discussed, but I think we need
> something. In my approach I tried to come up with one section
> ("relationship") that is the place for specifying all sorts of linke,
> dependency being one, just to come up with one extensible way of expressing
> things.
> Anyway, having the ability to manage dependencies by Heat seems necessary.
> And I would not pass the ball completely to the other tools outside of
> Heat. First of all, doing things in those other tools also gets complicated
> (e.g. while chef is good on one server, doing synchronization across
> servers can get ugly). And Heat has the ultimate knowledge about what
> servers it created, their IP addresses etc, so it should lead
> orchestration.
See the second depends_on bullet point here
https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config#Why_not_relationships_hosted_on.2C_depends_on.2C_connects_to.3F
I'll paste the content here:
depends_on is represented by the order of the components specified in
the components property of the OS::Nova::Server. Practically speaking,
one component does not "depend on" another at all. Instead one component
might have some precondition which could be satisfied by another
specific component, or by a different component which performs that
precondition in a different way, or that precondition might already be
satisfied by the server image. Modeling the satisfaction of these
preconditions is a core purpose of many configuration management tools.
Doing this in HOT components would be reinventing something which is
more completely implemented by the very tools that some components will
be using. Therefore this proposal takes the position that component
dependencies should not be modeled at all, and template authors should
ensure themselves that preconditions of a component are satisfied either
through specifying component order, or by building custom images.

>
> Regarding "component_execution: async"
> Is this necessary? I think in any case, it should be possible to create
> infrastructure resources (servers) in parallel. Then only the component
> startup should be synchronized once the servers are up, and this should be
> the default behavior. I think this actually related to the dependency topic
> above. BTW, even component startup inside servers can be done in parallel
> unless components have dependencies on each other, so doing a component
> startup in the strict order as given in a list in the template is probably
> not necessary.
There is an argument for not having a component_execution property and
async being the default as soon as it is implemented. However some users
may prefer having slower stack launch times just so they don't need to
think of concurrency concerns ;)

As for allowing a server to configure components in parallel, if your
configuration is so slow that you're tempted to parallelise then you're
possibly doing it wrong.  The slowest task would be package installation
- package managers take exclusive locks so that won't parallelise. In
any case we would encourage custom images with packages preloaded. That
leaves writing configuration files and restarting services - this should
be fast enough to run in whatever order is specified in OS::Nova::Server.
> Regarding the wait condition example:
> I get the idea and it surely would work, but I think it is still
> un-intuitive and we should about a more abstract declarative way for
> expressing such use cases.
Maybe, but there should still be an easier way of consuming
waitconditions - not necessarily resembling that example.
> Regarding the native tool bootstrap config proposal:
> I agree with other comments already made on this thread that the sheer
> number of different config components seems to much. I guess for users it
> will be hard to understand, which one to use when, what combination makes
> sense, in which order they have to be combined etc. Especially, when things
> are getting combined, my gut feeling is that the likelyhood of templates
> breaking whenever some of the underlying implementation changes will
> increase.
The only reason to support multiple tools is that potential users have
often already invested effort into the scripts for their favorite tool,
and would likely balk at trying heat if they had to rewrite all of their
software configuration in a different syntax. There will be
documentation guiding the user on what are the most appropriate
component types for a given situation.
> Steve Baker <sbaker at redhat.com> wrote on 16.10.2013 00:48:53:
>> From: Steve Baker <sbaker at redhat.com>
>> To: openstack-dev at lists.openstack.org,
>> Date: 16.10.2013 00:51
>> Subject: [openstack-dev] [Heat] HOT Software configuration proposal
>>
>> I've just written some proposals to address Heat's HOT software
>> configuration needs, and I'd like to use this thread to get some
> feedback:
>> https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config
>>
> https://wiki.openstack.org/wiki/Heat/Blueprints/native-tools-bootstrap-config
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131017/4e40402a/attachment.html>