[openstack-dev] [TripleO][Heat] Tuskar v. Heat responsibilities

Steven Hardy shardy at redhat.com
Thu Jun 25 21:40:11 UTC 2015

On Tue, Jun 23, 2015 at 04:05:08PM -0400, Jay Dobies wrote:
> I didn't want to hijack Steve Hardy's thread about the recursive validation,
> but I wanted to summarize the needs that Tuskar and the UI have been trying
> to answer and some of the problems we ran into.
> I think it's fairly common knowledge now that Tuskar and the THT templates
> diverged over the past few months, so I won't rehash it. If you need a
> summary of what happened, look here:
> https://jdob.fedorapeople.org/tuskar-heat.jpg

Haha, I think that's a fair analysis, although it's also true that
functionality in heat and our understanding of requirements has changed a
fair bit since Tuskar started out :)

> Below are some of the needs that the Tuskar UI in general has when working
> with the TripleO Heat Templates. I'm hoping we can come up with a decent
> list and use that to help drive what belongs in Heat v. what belongs
> elsewhere, and ultimately what that "elsewhere" actually is.

Definitely, and also I've increasingly come to the realization that when
TripleO needs something, it's normally because it's a generally needed
thing, e.g having a deployment specific composition API is probably a
mistake long-term and we do need to figure out a more general way to solve
these problems.

> = Choosing Component Implementations =
> == Background ==
> I'm already off to a bad start, since the word "component" isn't actually a
> term in this context. What I'm referring to is the fact that we are starting
> to see what is almost a plugin model in the THT templates.
> Previously, we had assumed that all of the overcloud configuration would be
> done through parameters. This is no longer the case as the resource_registry
> is used to add certain functionality.
> For example, in overcloud-resource-registry-puppet.yaml, we see:
>  # set to controller-config-pacemaker.yaml to enable pacemaker
>  OS::TripleO::ControllerConfig: puppet/controller-config.yaml
> That's a major overcloud configuration setting, but that choice isn't made
> through a parameter. It's in a different location and a different mechanism
> entirely.
> Similarly, enabling a Netapp backend for Cinder is done by setting a
> resource_registry entry to change the CinderBackend template [1]. This is a
> slightly different case conceptually than HA since the original template
> being overridden is a noop [2], but the mechanics of how to set it are the
> same.
> There are also a number of pre and post hooks that exist in the overcloud
> template that we are seeing more and more implementations of. RHEL
> registration is implemented as such a hook [3].
> I'm drawing a difference here between fundamental configuration changes (HA
> v. non-HA) and optional additions (RHEL registration). Again, mechanically
> they are implemented as resource_registry substitutions, though from a UI
> standpoint we'd likely want to treat them differently. Whether or not that
> difference is actually captured by the templates themselves or is purely in
> the UI is open to debate.
> == Usage in TripleO ==
> All of the examples I mentioned above have landed upstream and the Heat
> features necessary to facilitate them all exist.
> What doesn't exist is a way to manipulate the resource_registry. Tuskar
> doesn't have APIs for that level of changes; it assumed all configuration
> changes would be through parameters and hasn't yet had time to add in
> support for dorking with the registry in this fashion.
> While, technically, all of the resource_registry entries can be overridden,
> there are only a few that would make sense for a user to want to configure
> (I'm not talking about advanced users writing their own templates).
> On top of that, only certain templates can be used to fulfill certain
> resource types. For instance, you can't point CinderBackend to
> rhel-registration.yaml. That information isn't explicitly captured by Heat
> templates. I suppose you could inspect usages of a resource type in
> overcloud to determine the "api" of that type and then compare that to
> possible implementation templates' parameter lists to figure out what is
> compatible, but that seems like a heavy-weight approach.
> I mention that because part of the user experience would be knowing which
> resource types can have a template substitution made and what possible
> templates can fulfill it.

This is an interesting observation - assuming we wanted to solve this in
Heat (and I'm not saying we necessarily should), one can imagine a
resource_registry feature which works like constraints do for parameters,

    type: string
    - allowed_values: [dog, cat]

We could do likewise in the environment:

  OS::TripleO::ControllerConfig: puppet/controller-config.yaml
    - allowed_values:
      - puppet/controller-config.yaml,
      - foo/other-config.yaml]

These constraints would be enforced at stack validation time such that the
environment would be rejected if the optional constraints were not met.

That leaves the problem of encoding "this is the resource which selects the
CinderBackends", which I currently think can be adequately expressed via
resource namespacing/naming?

> == Responsibility ==
> Where should that be implemented? That's a good question.
> The idea of resolving resource type uses against candidate template
> parameter lists could fall under the model Steve Hardy is proposing of
> having Heat do it (he suggested the validate call, but this may be leading
> us more towards template inspection sorts of APIs supported by Heat.
> It is also possibly an addition to HOT, to somehow convey an interface so
> that we can more easily programatically look at a series of templates and
> understand how they play together. We used to be able to use the
> resource_registry to understand those relationships, but that's not going to
> work if we're trying to find substitutions into the registry.
> Alternatively, if Heat/HOT has no interest in any of this, this is something
> that Tuskar (or a Tuskar-like substitute) will need to solve going forward.

I think this problem of template composition is a general one, and I'm keen
to at least partially solve it via some (hopefully relatively simple and
incremental) additions to heat.

Clearly there's still plenty of scope for "application builder" type API's
on top of any new iterfaces, and I'm not trying to solve that problem,
but by exposing some richer interfaces around both template
inspection/validation and composition, hopefully we make such tasks

We *could* add some sort of interface schema to HOT, such as that
referenced in the TOSCA spec by Thomas in the other thread, right now I'm
just not fully convinced it's necessarily needed or justified - open to
further discussion/persuasion but I'm leaning towards keeping HOT simple

> = Consolidated Parameter List =
> == Background ==
> This is what Steve was getting at in his e-mail. I'll rehash the issue
> briefly.
> We used to be able to look at the parameters list in the overcloud template
> and know all of the parameters that need to be specified to configure the
> overcloud.
> The parameter passing is pretty strict, so if overcloud passes A, B, and C
> into a nested stack, the template for that nested stack must accept A, B,
> and C in its parameter list (others can be added and defaulted, but that
> doesn't totally answer the issue).
> So if I need D, E, and F in a particular implementation of a resource type
> (from above, think Netapp Cinder backend v. the default), they can't be
> passed like normal parameters. Those "normal" parameters are the least
> common denominator parameters that should be passed to a Cinder backend.
> It's similar to a common API into a plugin and a mechanism to pass custom
> configuration per plugin.
> In this case, that mechanism is to specify the parameter values in the
> parameter_defaults section of an environment file. The problem there is that
> the user still needs to somehow know the full set of these parameters which
> requires a lot of manual inspection of both the resource_registry (to
> understand all of the nested templates in play) and those templates (for
> their parameters above and beyond the ones passed by the overcloud
> template). There is no explicit schema for these sorts of extra parameters;
> it's all determined by through inspection.

Basically, this boils down to a heat validation pass which exposes any
parameters not defined by the parent template, which I think should be
possible via the recursive validation approach I outlined in my other mail,
do we have sufficient consensus to raise a spec for that and potentially
work up a PoC patch (I can do this)?

> == Usage in TripleO ==
> The UI needs to know this full set of parameters so it can present them to
> the user. It also needs some information to display them cleanly, meaning
> being able to co-locate related parameters (by "related" here I mostly mean
> "by service", but it's possible the UI people have additional ideas).

So I think this is kinda the point of parameter_groups, so if we were to
make better use of those over t-h-t, would that solve this problem?

> I made a comment on Steve's thread that the user should be told which
> parameters are specifically meant to be set via parameter_defaults so that
> it doesn't duplicate asking the user for them after already asking the user
> for the parameter that was defined in overcloud.

Yep, part of the validation part has to distinguish between:

1. Parameters already provided by parent
2. Parameters not provided by parent but with defaults (optional)
3. Parameters not provided by parent with no defaults (required)

(1) will be the "top level" parameters you always feed in to the parent
template, the lowest common denominator required by all implementations

(2) will be a set of defaults, provided either via parameter_defaults or
defaults directly in the templates.

(3) Are obviously those which the UI will always have to prompt for in
order to enable a successful stack-create

> You can really spin this out of control if you entertain the possibility
> that multiple nested templates can conflict on name if they use the same
> parameter name v. the situation where they'd explicitly *want* to use the
> same value. But we'll get there :)

Yeah, this is a problem, but it could be solve with either a naming
convention (combined with a "strict" mode of the validation pass which
rejects conflicting parameters of category (3).

Or, we could figure out a way to wire in per-resource parameter_defaults,
e.g similar to the syntax already supported for both hooks and type


      "OS::DBInstance": file:///home/mine/all_my_cool_templates/db.yaml
      hooks: pre-create
        aparam: 123

> == Responsibility ==
> Steve is making the argument that it belongs in Heat in the other thread. I
> think it's a useful feature, but  I don't want to discuss specifics and
> detract from his thread (we can continue over there). My intention is more
> to describe what the UI will want to do and why this is needed.
> = Saving Undeployed Stack Configuration =
> == Background ==
> One of the things Tuskar does is save the user values for an overcloud
> without making them live in Heat. I think this will continue to be needed
> for auditing, accountability, and ack workflows.
> Tuskar also supports versioning of a plan's configuration. For instance, the
> user could make a change, make the stack update, and find something is
> broken. Tuskar would provide a way to roll back the stack configuration to a
> previous version.

So, one of the reasons I'm pushing to get all this into the
heat environment, is because the initial idea of the environment was to
contain all of the, uh, environment specific data for your stack.

Taken to it's logical conclusion (modulo the gaps we discuss above), all
you need to save the user values is one (or more) environment files.

Saving and versioning/rollback of simple text files is *such* a solved
problem, I just don't see any justification for another service, if we can
encapsulate all the needed data in an environment definition.

For example, a UI prompts for various options, gets user parameters, then
pushes an environment file to swift, or git, or whatever :)

So the steps become:

1. Ask questions to generate initial environment file (structural options)
2. Validate it with heat
3. Potentially ask some more questions (site specific options)
4. Ask if you want to save or deploy (push to swift or heat stack-create).

Where *this* logic lives is probably the question, but to me it seems like
something that could be a Ux library rather than necessarily needing an API?

> == Responsibility ==
> From what I understand, Heat doesn't have any interest in storing plan
> parameters in this fashion. Though that comes from a while ago, so it's
> possible direction has changed.
> Otherwise, this one likely still falls in Tuskar. It's possible it's done
> through client-side calls directly to some sort of storage, but I really
> don't like the idea of having that much logic tied to a client (more
> specifically, tied into a Python client in the event that shops looking to
> integrate aren't using Python).

This is true, Heat doesn't want to store stack definitions other than for
live stacks, backups during update, and snapshots.

As outlined above, I think there are many (better) ways to store a series
of text files than inside heat, which is one reason why we've resisted
these sorts of "catalog" requirements in the past.

I don't really think it's tied to a client, if we can shape things such
that it's just pushing some collection of data to e.g swift after following
a well defined series of pre-deployment validation steps.

> = Conclusion =
> There might be some other lessons learned and I'll send another e-mail if I
> think of others, but those are what immediately jump out at me (this got
> kinda long and I need to cut it off).
> There is probably a better way of continuing the discussion than replying
> inline to this wall of text. But for now, I wanted to at least throw this
> out there as Steve Hardy and others start to talk about composabilty.

Yeah this is great, thanks for starting the discussion - it's starting to
feel like there are some potentially definable features falling out of the
discussion, will be interested to see if you're in agreement or not :)


More information about the OpenStack-dev mailing list