Open Stack

On 06/22/2015 12:19 PM, Steven Hardy wrote:
> Hi all,
>
> Lately I've been giving some thought to how we might enable easier
> composability, and in particular how we can make it easier for folks to
> plug in deeply nested optional extra logic, then pass data in via
> parameter_defaults to that nested template.
>
> Here's an example of the use-case I'm describing:
>
> https://review.openstack.org/#/c/193143/5/environments/cinder-netapp-config.yaml
>
> Here, we want to allow someone to easily turn on an optional configuration
> or feature, in this case a netapp backend for cinder.

I think the actual desired goal is bigger than just optional 
configuration. I think it revolves more around choosing a nested stack 
implementation for a resource type and how to manage custom parameters 
for that implementation. We're getting into the territory here of having 
a parent stack defining an API that nested stacks can plug into. I'd 
like to have some sort of way of deriving that information instead of 
having it be completely relegated to outside documentation (but I'm 
getting off topic; at the end I mention how I want to do a better write 
up of the issues Tuskar has faced and I'll elaborate more there).

> The parameters specific to this feature/configuration only exist in the
> nested cinder-netapp-config.yaml template, then parameter_defaults are used
> to wire in the implementation specific data without having to pass the
> values through every parent template (potentially multiple layers of
> nesting).
>
> This approach is working out OK, but we're missing an interface which makes
> the schema for parameters over the whole tree available.
 >
> This is obviously
> a problem, particularly for UI's, where you really need a clearly defined
> interface for what data is required, what type it is, and what valid values
> may be chosen.

I think this is going to be an awesome addition to Heat. As you alluded 
to, we've struggled with this in TripleO. The parameter_defaults works 
to circumvent the parameter passing, but it's rough from a user 
experience point of view since getting the unified list of what's 
configurable is difficult.

> I'm considering an optional additional flag to our template-validate API
> which allows recursive validation of a tree of templates, with the data
> returned on success to include a tree of parameters, e.g:
>
> heat template-validate -f parent.yaml -e env.yaml --show-nested
> {
>    "Description": "The Parent",
>    "Parameters": {
>      "ParentConfig": {
>        "Default": [],
>        "Type": "Json",
>        "NoEcho": "false",
>        "Description": "",
>        "Label": "ExtraConfig"
>      },
>      "ControllerFlavor": {
>        "Type": "String",
>        "NoEcho": "false",
>        "Description": "",
>        "Label": "ControllerFlavor"
>      }
>    }
>   "NestedParameters": {
>      "child.yaml": {
>          "Parameters": {
>            "ChildConfig": {
>            "Default": [],
>            "Type": "Json",
>            "NoEcho": "false",
>            "Description": "",
>            "Label": "Child ExtraConfig"
>            }
>          }
>       }
> }

Are you intending on resolving parameters passed into a nested stack 
from the parent against what's defined in the nested stack's parameter 
list? I'd want NestedParameters to only list things that aren't already 
being specified to the parent.

Specifically with regard to the TripleO Heat templates, there is still a 
lot of logic that needs to be applied to properly divide out parameters. 
For example, there are some things passed in from the parents to the 
nested stacks that are kinda namespaced by convention, but its not a 
hard convention. So to try to group the parameters by service, we'd have 
to look at a particular NestedParameters section and then also add in 
anything from the parent that applies to that service. I don't believe 
we can use parameter groups to correlate them (we might be able to, or 
that might be its own improvement).

I realize that's less of a Heat issue and more of a THT issue, but I 
figured I'd bring it up anyway.

> This implies that we also need to pass the "files" map to the validate API,
> like we currently do for create (we already pass the environment, although
> it's not really doing much beyond providing parameters for the parent stack
> AFAICT, we completely skip validating TemplateResources because the files
> aren't passed):
>
> https://github.com/openstack/heat/blob/master/heat/engine/service.py#L873
>
> Before I go ahead and spend time writing a spec/code for this, what do
> folks think about enhancing validate like this?  Is there an alternative,
> for example adding a parameters schema output to stack-preview?

For what it's worth, I'd rather see this as a spec before code. There 
are a lot of complications we hit in Tuskar in trying to make 
configuring the overcloud through THT user-friendly. This is one part of 
it, but there are others. I'd like to have them all talked out and see 
what the larger group of changes are.

For example, take the cinder-netapp-config example you mentioned. That 
can only be used to fulfill a specific resource type in the registry. 
That knowledge isn't captured anywhere, so it relies on some sort of 
external documentation saying "choose from these two child stacks for X, 
choose from these three for Y". That's something that Tuskar going 
forward is going to need to capture, but that also might be another 
candidate for adding to Heat and helping the user experience for complex 
nested stacks in general.

Perhaps this is another argument for a schema of sorts, which is why I'd 
like to see us lay all of these issues out at the same time in specs 
before any code is written on a particular implementation. I'll try to 
find time in the next few days to do a better write up of the kinds of 
questions that Tuskar had to try to answer about the THT templates.

> Also interested in TripleO feedback on this, particularly to what extent do
> we think this could potentially replace some of the template introspection
> currently done via Tuskar?
>
> Thanks,
>
> Steve
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>