[openstack-dev] [tripleo] When to use parameters vs parameter_defaults

Jay Dobies jason.dobies at redhat.com
Mon Nov 23 12:58:07 UTC 2015


On 11/20/2015 07:05 PM, Ben Nemec wrote:
> Thinking about this some more makes me wonder if we need a sample config
> generator like oslo.config.  It would work off something similar to the
> capabilities map, where you would say
>
> SSL:
>    templates:
>      -puppet/extraconfig/tls/tls-cert-inject.yaml
>    output:
>      -environments/enable-ssl.yaml
>
> And the tool would look at that, read all the params from
> tls-cert-inject.yaml and generate the sample env file.  We'd have to be
> able to do a few new things with the params in order for this to work:
>
> -Need to specify whether a param is intended to be set as a top-level
> param, parameter_defaults (which we informally do today with the Can be
> overridden by parameter_defaults comment), or internal, to define params
> that shouldn't be exposed in the sample config and are only intended as
> an interface between templates.  There wouldn't be any enforcement of
> the internal type, but Python relies on convention for its private
> members so there's precedent. :-)

There is new functionality in Heat that will let you pass in a series of 
templates and environments and it will return:

- The list of top-level parameters, the same way template-validate 
always did
- A list of all nested parameters, keyed by resource.

Take a look at 
https://github.com/openstack/heat-specs/blob/master/specs/liberty/nested-validation.rst 
for details and an example.

That's not entirely what you're getting at, I realize that. I'm glad to 
see you suggest a convention-based approach because I think that's the 
only way we're going to be able to convey some of this information.

I think at the same time we add a mechanism to distinguish between 
internal and external parameters, we need to add something to indicate 
required v. optional.

With a nested stack, anything that's not part of the top-level parameter 
contract is defaulted. The problem is that it loses information on what 
is a valid default v. what's simply defaulted to pass validation.

I've been noticing this more and more on the vendor integrations. They 
have parameters that are required (such as a username) and others that 
are less likely to be changed (I can't think of an example, but I think 
everyone can see where I'm going with this).

So I think there are two sorts of things (at least, I'm also thinking 
off the top of my head) we'd like this tool/sample file to convey:

- Parameters a user would want to change, as compared to those used for 
internal data shuffling
- Information on if the user must supply a value, as compared to 
parameters with an actual default

All that said, I dig this idea of a tool that would generate a skeleton 
environment file.

> -There would have to be some way to pick out only certain params from a
> template, since I think there are almost certainly features that are
> configured using a subset of say puppet/controller.yaml which obviously
> can't just take the params from an entire file.  Although maybe this is
> an indication that we could/should refactor the templates to move some
> of these optional params into their own separate files (at this point I
> think I should take a moment to mention that this is somewhat of a brain
> dump, so I haven't thought through all of the implications yet and I'm
> not sure it all makes sense).

> The nice thing about generating these programmatically is we would
> formalize the interface of the templates somewhat, and it would be
> easier to keep sample envs in sync with the actual implementation.

You could go so far as to put CI on top of it like we do with the oslo 
config stuff, which would be neat.

> You'd never have to worry about someone adding a param to a file but
> forgetting to update the env (or at least it would be easy to catch and
> fix when they did, just run "tox -e genconfig").
>
> I'm not saying this is a simple or short-term solution, but I'm curious
> what people think about setting this as a longer-term goal, because as I
> think our discussion in Tokyo exposed, we're probably going to have a
> bit of an explosion of sample envs soon and we're going to need some way
> to keep them sane.
>
> Some more comments inline.
>
> On 11/19/2015 10:16 AM, Steven Hardy wrote:
>> On Mon, Nov 16, 2015 at 08:15:48PM +0100, Giulio Fidente wrote:
>>> On 11/16/2015 04:25 PM, Steven Hardy wrote:
>>>> Hi all,
>>>>
>>>> I wanted to start some discussion re $subject, because it's been apparrent
>>>> that we have a lack of clarity on this issue (and have done ever since we
>>>> started using parameter_defaults).
>>>
>>> [...]
>>>
>>>> How do people feel about this example, and others like it, where we're
>>>> enabling common, but not mandatory functionality?
>>>
>>> At first I was thinking about something as simple as: "don't use top-level
>>> params for resources which the registry doesn't enable by default".
>>>
>>> It seems to be somewhat what we tried to do with the existing pluggable
>>> resources.
>>>
>>> Also, not to hijack the thread but I wanted to add another question related
>>> to a similar issue:
>>>
>>>    Is there a reason to prefer use of parameters: instead of
>>> parameter_defaults: in the environment files?
>>>
>>> It looks to me that by defaulting to parameter_defaults: users won't need to
>>> update their environment files in case the parameter is moved from top-level
>>> into a specific nested stack so I'm inclined to prefer this. Are there
>>> reasons not to?
>>
>> The main reason is scope - if you use "parameters", you know the data flow
>> happens via the parent template (e.g overcloud-without-mergepy) and you
>> never have to worry about naming collisions outside of that template.
>>
>> But if you use parameter_defaults, all parameters values defined that way
>> are effectively global, and you then have to be much more careful that you
>> never shadow a parameter name and get an unexpected value passed in to it.
>>
>> Here's another example of why we need to decide this btw:
>>
>> https://review.openstack.org/#/c/229471/
>>
>> Here, we have some workers parameters, going only into controller.yaml -
>> this is fine, but the new options are completely invisible to users who
>> look at the overcloud_without_mergepy parameters schema as their interface
>> (in particular I'm thinking of any UI here).
>>
>> My personal preference is to say:
>>
>> 1. Any templates which are included in the default environment (e.g
>> overcloud-resource-registry-puppet.yaml), must expose their parameters
>> via overcloud-without-mergepy.yaml
>>
>> 2. Any templates which are included in the default environment, but via a
>> "noop" implementation *may* expose their parameters provided they are
>> common and not implementation/vendor specific.
>
> This seems like a reasonable approach, although that "may" still leaves
> a lot of room for bikeshedding. ;-)
>
> It might be good to say that in this case it is "preferred" to use a
> top-level param, but if there's a reason not to then it's acceptable to
> use parameter_defaults.  An example for the SSL case would be the
> certificate path - I specifically do not want that visibly exposed to
> the user at this point, so I wouldn't want it added to the top-level
> template.  I consider that an implementation detail where if you know
> what you're doing you can override it, but otherwise you shouldn't touch it.
>
>>
>> 3. Any templates exposing vendor specific interfaces (e.g at least anything
>> related to the OS::TripleO::*ExtraConfig* interfaces) must not expose any
>> parameters via the top level template.
>>
>> How does this sound?
>>
>> This does mean we suffer some template bloat from (1) and (2), but it makes
>> the job of any UI or other tool requiring user input much easier, I think?
>>
>> Steve
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



More information about the OpenStack-dev mailing list