Open Stack

On Fri, 2015-11-20 at 16:53 +0100, Jiri Tomasek wrote:
> On 11/16/2015 04:25 PM, Steven Hardy wrote:
> > Hi all,
> > 
> > I wanted to start some discussion re $subject, because it's been
> > apparrent
> > that we have a lack of clarity on this issue (and have done ever
> > since we
> > started using parameter_defaults).
> > 
> > Some context:
> > 
> > - Historically TripleO has provided a fairly comprehensive "top
> > level"
> >    parameters interface, where many per-role and common options are
> >    specified, then passed in to the respective ResourceGroups on
> > deployment
> > 
> > https://git.openstack.org/cgit/openstack/tripleo-heat-templates/tre
> > e/overcloud-without-mergepy.yaml#n14
> > 
> > The nice thing about this approach is it gives a consistent API to
> > the
> > operator, e.g the parameters schema for the main overcloud template
> > defines
> > most of the expected inputs to the deployment.
> > 
> > The main disadvantage is a degree of template bloat, where we wire
> > dozens
> > of parameters into each ResourceGroup, and from there into whatever
> > nested
> > templates consume them.
> > 
> > - When we started adding interfaces (such as all the
> > OS::TripleO::*ExtraConfig*
> >    interfaces, there was a need to enable passing arbitrary
> > additional
> >    values to nested templates, with no way of knowing what they are
> > (e.g to
> >    enable wiring in third-party pieces we have no knowledge of or
> > which
> >    require implementation-specific arguments which don't make sense
> > for all
> >    deployments.
> > 
> > To do this, we made use of the heat parameter_defaults interface,
> > which
> > (unlike normal parameters) have global scope (visible to all nested
> > stacks,
> > without explicitly wiring in the values from the parent):
> > 
> > http://docs.openstack.org/developer/heat/template_guide/environment
> > .html#define-defaults-to-parameters
> > 
> > The nice thing about this approach is its flexibility, any
> > arbitrary
> > values can be provided without affecting the parent templates, and
> > it can
> > allow for a terser implementation because you only specify the
> > parameter
> > definition where it's actually used.
> > 
> > The main disadvantage of this approach is it becomes very much
> > harder to
> > discover an API surface for the operator, e.g the parameters that
> > must be
> > provided on deployment by any CLI/UI tools etc.  This has been
> > partially
> > addressed by the new-for-liberty nested validation heat feature,
> > but
> > there's still a bunch of unsolved complexity around how to actually
> > consume
> > that data and build a coherent consolidated API for user
> > interaction:
> > 
> > https://github.com/openstack/heat-specs/blob/master/specs/liberty/n
> > ested-validation.rst
> > 
> > My question is, where do we draw the line on when to use each
> > interface?
> > 
> > My position has always been that we should only use
> > parameter_defaults for
> > the ExtraConfig interfaces, where we cannot know what reasonable
> > parameters
> > are.  And for all other "core" functionality, we should accept the
> > increased
> > template verbosity and wire arguments in from overcloud-without-
> > mergepy.
> > 
> > However we've got some patches which fall into a grey area, e.g
> > this SSL
> > enablement patch:
> > 
> > https://review.openstack.org/#/c/231930/46/overcloud-without-mergep
> > y.yaml
> > 
> > Here we're actually removing some existing (non functional) top-
> > level
> > parameters, and moving them to parameter_defaults.
> > 
> > I can see the logic behind it, it does make the templates a bit
> > cleaner,
> > but at the expense of discoverablility of those (probably not
> > implementation dependent) parameters.
> > 
> > How do people feel about this example, and others like it, where
> > we're
> > enabling common, but not mandatory functionality?
> > 
> > In particular I'm keen to hear from Mainn and others interested in
> > building
> > UIs on top of TripleO as to which is best from that perspective,
> > and how
> > such arguments may be handled relative to the capabilities mapping
> > proposed
> > here:
> > 
> > https://review.openstack.org/#/c/242439/
> > 
> > Thanks!
> > 
> > Steve
> > 
> > ___________________________________________________________________
> > _______
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsu
> > bscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> I think I'll try to do a bit of a recap to make sure I understand 
> things. It may shift slightly off the topic of this thread but I
> think 
> it is worth it and it will describe what the GUI is able/expecting to
> work with.
> 
> Template defines parameters and passes them to child templates via 
> resource properties.
> Root template parameter values are set by (in order of precedence):
> 1. 'parameters' param in 'stack create' api call or 'parameters'
> section 
> in environment
> 2. 'parameter_defaults' section in environment
> 3. 'default' in parameter definition in template
> 
> Non-root template parameter values are set by (in order of
> precedence):
> 1. parent resource properties
> 2. 'parameter_defaults' in environment
> 3. 'default' in parameter definition in template
> 
> The name collisions in parameter_defaults should not be a problem
> since 
> the template author should make sure, the parameters names he defines
> don't collide with other templates.


We could actually write a unit test in tripleo-heat-templates that ran
in the gate that parsed all .yaml files, created a map... and verified
that perhaps the descriptions matched (exactly). Perhaps a bit pedantic
but it might help us avoid potential overlap in nested stacks which
can't be wired in at the top level. (thinking of some of the network
isolation stuff).

Dan

> 
> The GUI's main goal (same as CLI and tripleo-common) is not to
> hardcode 
> anything and use THT (or any other set of files) which user provides
> as 
> a source of all information. It prepares deployment in several steps:
> 1. Plan Creation - uploads THT files and specifies root template.
> 
> 2. Environments selection - based on Capabilities map which is
> provided 
> as part of THT (https://review.openstack.org/#/c/242439/) user picks 
> which environments he intends to use for deployment.
> 
> 3. Parameters setting - Based on 2. tripleo-common runs heat template
> validation and input for GUI is nested parameters json structure. The
> output from GUI (through tripleo-common) is a new 'temporary' 
> environment which holds values in 'parameters' section for root
> template 
> parameters and values for nested parameters (without 'value') in 
> 'parameter_defaults' section.
> 
> This means that in GUI we'll provide form inputs for every parameter 
> from the root template and for the nested parameters which don't have
> a 
> 'value' attribute.
> The inputs will be pre-filled with 'default' value if parameter has
> one 
> provided (either from param_defaults or parameter definition).
> 
> I'll need to do more thinking about how what I describe here is
> affected 
> by what we are trying to solve in $subject...
> 
> 4. Deploy  - for deployment, all environments are merged: Root 
> Environment < Custom Environments < Temporary Environment.
> 
> 
> Resource capabilities
> I have put some thinking into resource capabilities spec usefulness
> when 
> I was creating the Capabilities map. I think it is much more useful
> to 
> specify multiple Environments which are mutually exclusive and
> implement 
> some functionality's configuration. It seems to be much better to
> make 
> the choice on the environment level rather then resource_registry
> level, 
> because the selection of templates possible to use for implementing
> some 
> resource changes with the environments selected. So resource 
> capabilities depend on the Environments selected. So to make really 
> simple example, if we have a resource, which can be implemented by 2 
> templates, I'd create 2 environments which defines resource_registry 
> each with one of the templates available.
> 
> What I mean by this is, it is better to let user choose on the 
> environment level rather than on resource_registry level. Resource 
> capabilities could be valid in terms of providing validation that 
> certain template can be used to implement certain resource. Although 
> this seems not needed as the creator of Environment should make sure
> he 
> uses template which is able to implement that resource.
> 
> I should probably add this to the resource capabilities spec review.
> 
> 
> Jirka
> 
> _____________________________________________________________________
> _____
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubs
> cribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev