Open Stack

I didn't want to hijack Steve Hardy's thread about the recursive 
validation, but I wanted to summarize the needs that Tuskar and the UI 
have been trying to answer and some of the problems we ran into.

I think it's fairly common knowledge now that Tuskar and the THT 
templates diverged over the past few months, so I won't rehash it. If 
you need a summary of what happened, look here: 
https://jdob.fedorapeople.org/tuskar-heat.jpg

Below are some of the needs that the Tuskar UI in general has when 
working with the TripleO Heat Templates. I'm hoping we can come up with 
a decent list and use that to help drive what belongs in Heat v. what 
belongs elsewhere, and ultimately what that "elsewhere" actually is.


= Choosing Component Implementations =

== Background ==

I'm already off to a bad start, since the word "component" isn't 
actually a term in this context. What I'm referring to is the fact that 
we are starting to see what is almost a plugin model in the THT templates.

Previously, we had assumed that all of the overcloud configuration would 
be done through parameters. This is no longer the case as the 
resource_registry is used to add certain functionality.

For example, in overcloud-resource-registry-puppet.yaml, we see:

  # set to controller-config-pacemaker.yaml to enable pacemaker
  OS::TripleO::ControllerConfig: puppet/controller-config.yaml

That's a major overcloud configuration setting, but that choice isn't 
made through a parameter. It's in a different location and a different 
mechanism entirely.

Similarly, enabling a Netapp backend for Cinder is done by setting a 
resource_registry entry to change the CinderBackend template [1]. This 
is a slightly different case conceptually than HA since the original 
template being overridden is a noop [2], but the mechanics of how to set 
it are the same.

There are also a number of pre and post hooks that exist in the 
overcloud template that we are seeing more and more implementations of. 
RHEL registration is implemented as such a hook [3].

I'm drawing a difference here between fundamental configuration changes 
(HA v. non-HA) and optional additions (RHEL registration). Again, 
mechanically they are implemented as resource_registry substitutions, 
though from a UI standpoint we'd likely want to treat them differently. 
Whether or not that difference is actually captured by the templates 
themselves or is purely in the UI is open to debate.

== Usage in TripleO ==

All of the examples I mentioned above have landed upstream and the Heat 
features necessary to facilitate them all exist.

What doesn't exist is a way to manipulate the resource_registry. Tuskar 
doesn't have APIs for that level of changes; it assumed all 
configuration changes would be through parameters and hasn't yet had 
time to add in support for dorking with the registry in this fashion.

While, technically, all of the resource_registry entries can be 
overridden, there are only a few that would make sense for a user to 
want to configure (I'm not talking about advanced users writing their 
own templates).

On top of that, only certain templates can be used to fulfill certain 
resource types. For instance, you can't point CinderBackend to 
rhel-registration.yaml. That information isn't explicitly captured by 
Heat templates. I suppose you could inspect usages of a resource type in 
overcloud to determine the "api" of that type and then compare that to 
possible implementation templates' parameter lists to figure out what is 
compatible, but that seems like a heavy-weight approach.

I mention that because part of the user experience would be knowing 
which resource types can have a template substitution made and what 
possible templates can fulfill it.

== Responsibility ==

Where should that be implemented? That's a good question.

The idea of resolving resource type uses against candidate template 
parameter lists could fall under the model Steve Hardy is proposing of 
having Heat do it (he suggested the validate call, but this may be 
leading us more towards template inspection sorts of APIs supported by Heat.

It is also possibly an addition to HOT, to somehow convey an interface 
so that we can more easily programatically look at a series of templates 
and understand how they play together. We used to be able to use the 
resource_registry to understand those relationships, but that's not 
going to work if we're trying to find substitutions into the registry.

Alternatively, if Heat/HOT has no interest in any of this, this is 
something that Tuskar (or a Tuskar-like substitute) will need to solve 
going forward.


= Consolidated Parameter List =

== Background ==

This is what Steve was getting at in his e-mail. I'll rehash the issue 
briefly.

We used to be able to look at the parameters list in the overcloud 
template and know all of the parameters that need to be specified to 
configure the overcloud.

The parameter passing is pretty strict, so if overcloud passes A, B, and 
C into a nested stack, the template for that nested stack must accept A, 
B, and C in its parameter list (others can be added and defaulted, but 
that doesn't totally answer the issue).

So if I need D, E, and F in a particular implementation of a resource 
type (from above, think Netapp Cinder backend v. the default), they 
can't be passed like normal parameters. Those "normal" parameters are 
the least common denominator parameters that should be passed to a 
Cinder backend. It's similar to a common API into a plugin and a 
mechanism to pass custom configuration per plugin.

In this case, that mechanism is to specify the parameter values in the 
parameter_defaults section of an environment file. The problem there is 
that the user still needs to somehow know the full set of these 
parameters which requires a lot of manual inspection of both the 
resource_registry (to understand all of the nested templates in play) 
and those templates (for their parameters above and beyond the ones 
passed by the overcloud template). There is no explicit schema for these 
sorts of extra parameters; it's all determined by through inspection.

== Usage in TripleO ==

The UI needs to know this full set of parameters so it can present them 
to the user. It also needs some information to display them cleanly, 
meaning being able to co-locate related parameters (by "related" here I 
mostly mean "by service", but it's possible the UI people have 
additional ideas).

I made a comment on Steve's thread that the user should be told which 
parameters are specifically meant to be set via parameter_defaults so 
that it doesn't duplicate asking the user for them after already asking 
the user for the parameter that was defined in overcloud.

You can really spin this out of control if you entertain the possibility 
that multiple nested templates can conflict on name if they use the same 
parameter name v. the situation where they'd explicitly *want* to use 
the same value. But we'll get there :)

== Responsibility ==

Steve is making the argument that it belongs in Heat in the other 
thread. I think it's a useful feature, but  I don't want to discuss 
specifics and detract from his thread (we can continue over there). My 
intention is more to describe what the UI will want to do and why this 
is needed.


= Saving Undeployed Stack Configuration =

== Background ==

One of the things Tuskar does is save the user values for an overcloud 
without making them live in Heat. I think this will continue to be 
needed for auditing, accountability, and ack workflows.

Tuskar also supports versioning of a plan's configuration. For instance, 
the user could make a change, make the stack update, and find something 
is broken. Tuskar would provide a way to roll back the stack 
configuration to a previous version.

== Responsibility ==

 From what I understand, Heat doesn't have any interest in storing plan 
parameters in this fashion. Though that comes from a while ago, so it's 
possible direction has changed.

Otherwise, this one likely still falls in Tuskar. It's possible it's 
done through client-side calls directly to some sort of storage, but I 
really don't like the idea of having that much logic tied to a client 
(more specifically, tied into a Python client in the event that shops 
looking to integrate aren't using Python).


= Conclusion =

There might be some other lessons learned and I'll send another e-mail 
if I think of others, but those are what immediately jump out at me 
(this got kinda long and I need to cut it off).

There is probably a better way of continuing the discussion than 
replying inline to this wall of text. But for now, I wanted to at least 
throw this out there as Steve Hardy and others start to talk about 
composabilty.



[1] https://review.openstack.org/#/c/187700/

[2] 
https://review.openstack.org/#/c/187700/10/overcloud-resource-registry-puppet.yaml

[3] 
https://git.openstack.org/cgit/openstack/tripleo-heat-templates/tree/extraconfig/post_deploy/rhel-registration/rhel-registration-resource-registry.yaml