Open Stack

On 11/25/2015 09:13 AM, Dan Prince wrote:
> On Fri, 2015-11-20 at 16:53 +0100, Jiri Tomasek wrote:
>> On 11/16/2015 04:25 PM, Steven Hardy wrote:
>>> Hi all,
>>>
>>> I wanted to start some discussion re $subject, because it's been
>>> apparrent
>>> that we have a lack of clarity on this issue (and have done ever
>>> since we
>>> started using parameter_defaults).
>>>
>>> Some context:
>>>
>>> - Historically TripleO has provided a fairly comprehensive "top
>>> level"
>>>    parameters interface, where many per-role and common options are
>>>    specified, then passed in to the respective ResourceGroups on
>>> deployment
>>>
>>> https://git.openstack.org/cgit/openstack/tripleo-heat-templates/tre
>>> e/overcloud-without-mergepy.yaml#n14
>>>
>>> The nice thing about this approach is it gives a consistent API to
>>> the
>>> operator, e.g the parameters schema for the main overcloud template
>>> defines
>>> most of the expected inputs to the deployment.
>>>
>>> The main disadvantage is a degree of template bloat, where we wire
>>> dozens
>>> of parameters into each ResourceGroup, and from there into whatever
>>> nested
>>> templates consume them.
>>>
>>> - When we started adding interfaces (such as all the
>>> OS::TripleO::*ExtraConfig*
>>>    interfaces, there was a need to enable passing arbitrary
>>> additional
>>>    values to nested templates, with no way of knowing what they are
>>> (e.g to
>>>    enable wiring in third-party pieces we have no knowledge of or
>>> which
>>>    require implementation-specific arguments which don't make sense
>>> for all
>>>    deployments.
>>>
>>> To do this, we made use of the heat parameter_defaults interface,
>>> which
>>> (unlike normal parameters) have global scope (visible to all nested
>>> stacks,
>>> without explicitly wiring in the values from the parent):
>>>
>>> http://docs.openstack.org/developer/heat/template_guide/environment
>>> .html#define-defaults-to-parameters
>>>
>>> The nice thing about this approach is its flexibility, any
>>> arbitrary
>>> values can be provided without affecting the parent templates, and
>>> it can
>>> allow for a terser implementation because you only specify the
>>> parameter
>>> definition where it's actually used.
>>>
>>> The main disadvantage of this approach is it becomes very much
>>> harder to
>>> discover an API surface for the operator, e.g the parameters that
>>> must be
>>> provided on deployment by any CLI/UI tools etc.  This has been
>>> partially
>>> addressed by the new-for-liberty nested validation heat feature,
>>> but
>>> there's still a bunch of unsolved complexity around how to actually
>>> consume
>>> that data and build a coherent consolidated API for user
>>> interaction:
>>>
>>> https://github.com/openstack/heat-specs/blob/master/specs/liberty/n
>>> ested-validation.rst
>>>
>>> My question is, where do we draw the line on when to use each
>>> interface?
>>>
>>> My position has always been that we should only use
>>> parameter_defaults for
>>> the ExtraConfig interfaces, where we cannot know what reasonable
>>> parameters
>>> are.  And for all other "core" functionality, we should accept the
>>> increased
>>> template verbosity and wire arguments in from overcloud-without-
>>> mergepy.
>>>
>>> However we've got some patches which fall into a grey area, e.g
>>> this SSL
>>> enablement patch:
>>>
>>> https://review.openstack.org/#/c/231930/46/overcloud-without-mergep
>>> y.yaml
>>>
>>> Here we're actually removing some existing (non functional) top-
>>> level
>>> parameters, and moving them to parameter_defaults.
>>>
>>> I can see the logic behind it, it does make the templates a bit
>>> cleaner,
>>> but at the expense of discoverablility of those (probably not
>>> implementation dependent) parameters.
>>>
>>> How do people feel about this example, and others like it, where
>>> we're
>>> enabling common, but not mandatory functionality?
>>>
>>> In particular I'm keen to hear from Mainn and others interested in
>>> building
>>> UIs on top of TripleO as to which is best from that perspective,
>>> and how
>>> such arguments may be handled relative to the capabilities mapping
>>> proposed
>>> here:
>>>
>>> https://review.openstack.org/#/c/242439/
>>>
>>> Thanks!
>>>
>>> Steve
>>>
>>> ___________________________________________________________________
>>> _______
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsu
>>> bscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> I think I'll try to do a bit of a recap to make sure I understand 
>> things. It may shift slightly off the topic of this thread but I
>> think 
>> it is worth it and it will describe what the GUI is able/expecting to
>> work with.
>>
>> Template defines parameters and passes them to child templates via 
>> resource properties.
>> Root template parameter values are set by (in order of precedence):
>> 1. 'parameters' param in 'stack create' api call or 'parameters'
>> section 
>> in environment
>> 2. 'parameter_defaults' section in environment
>> 3. 'default' in parameter definition in template
>>
>> Non-root template parameter values are set by (in order of
>> precedence):
>> 1. parent resource properties
>> 2. 'parameter_defaults' in environment
>> 3. 'default' in parameter definition in template
>>
>> The name collisions in parameter_defaults should not be a problem
>> since 
>> the template author should make sure, the parameters names he defines
>> don't collide with other templates.
> 
> 
> We could actually write a unit test in tripleo-heat-templates that ran
> in the gate that parsed all .yaml files, created a map... and verified
> that perhaps the descriptions matched (exactly). Perhaps a bit pedantic
> but it might help us avoid potential overlap in nested stacks which
> can't be wired in at the top level. (thinking of some of the network
> isolation stuff).

+1.  This is how oslo.config works too - you can define the same option
in multiple places, but the descriptions have to match down to the
character.

> 
> Dan
> 
>>
>> The GUI's main goal (same as CLI and tripleo-common) is not to
>> hardcode 
>> anything and use THT (or any other set of files) which user provides
>> as 
>> a source of all information. It prepares deployment in several steps:
>> 1. Plan Creation - uploads THT files and specifies root template.
>>
>> 2. Environments selection - based on Capabilities map which is
>> provided 
>> as part of THT (https://review.openstack.org/#/c/242439/) user picks 
>> which environments he intends to use for deployment.
>>
>> 3. Parameters setting - Based on 2. tripleo-common runs heat template
>> validation and input for GUI is nested parameters json structure. The
>> output from GUI (through tripleo-common) is a new 'temporary' 
>> environment which holds values in 'parameters' section for root
>> template 
>> parameters and values for nested parameters (without 'value') in 
>> 'parameter_defaults' section.
>>
>> This means that in GUI we'll provide form inputs for every parameter 
>> from the root template and for the nested parameters which don't have
>> a 
>> 'value' attribute.
>> The inputs will be pre-filled with 'default' value if parameter has
>> one 
>> provided (either from param_defaults or parameter definition).
>>
>> I'll need to do more thinking about how what I describe here is
>> affected 
>> by what we are trying to solve in $subject...
>>
>> 4. Deploy  - for deployment, all environments are merged: Root 
>> Environment < Custom Environments < Temporary Environment.
>>
>>
>> Resource capabilities
>> I have put some thinking into resource capabilities spec usefulness
>> when 
>> I was creating the Capabilities map. I think it is much more useful
>> to 
>> specify multiple Environments which are mutually exclusive and
>> implement 
>> some functionality's configuration. It seems to be much better to
>> make 
>> the choice on the environment level rather then resource_registry
>> level, 
>> because the selection of templates possible to use for implementing
>> some 
>> resource changes with the environments selected. So resource 
>> capabilities depend on the Environments selected. So to make really 
>> simple example, if we have a resource, which can be implemented by 2 
>> templates, I'd create 2 environments which defines resource_registry 
>> each with one of the templates available.
>>
>> What I mean by this is, it is better to let user choose on the 
>> environment level rather than on resource_registry level. Resource 
>> capabilities could be valid in terms of providing validation that 
>> certain template can be used to implement certain resource. Although 
>> this seems not needed as the creator of Environment should make sure
>> he 
>> uses template which is able to implement that resource.
>>
>> I should probably add this to the resource capabilities spec review.
>>
>>
>> Jirka
>>
>> _____________________________________________________________________
>> _____
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubs
>> cribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>