Open Stack

On 08/25/2016 03:55 PM, Steven Hardy wrote:
> On Thu, Aug 25, 2016 at 04:19:12PM -0400, Zane Bitter wrote:
> > On 25/08/16 14:02, Steven Hardy wrote:
> >> Hi all,
> >>
> >> So I'm following up on a discussion that started here:
> >>
> >> https://review.openstack.org/#/c/356240
> >>
> >> Basically I recently suggested[1] that relaxing the restriction where a
> >> parameter can only exist in exactly one parameter_group would be a way to
> >> help work around some pain-points we have in TripleO atm.
> >
> > I usually read the scrollback of meetings I'm not able to attend but I must
> > have missed that one, sorry.
> >
> >> Let me start with the the problems we're trying to solve, because I think
> >> they are common to many Heat users, not just TripleO:
> >>
> >> 1. When doing nested validation to discover parameter schema, it's
> >> impossible to tell if a parameter will be provided by the parent
> >
> > IMHO they should all be provided by the parent, but that's another story ;)
> >
> >> This is a known issue from when nested validation was first implemented,
> >> and we never figured out a satisfactory fix, basically because you can't
> >> possibly tell without actually creating things whether an indirectly
> >> provided parameter (e.g a reference to another resource in a parent
> >> template) will resolve to a valid value.
> >>
> >> So when you build a bunch of templates where there are some parameters
> >> which form part of an internal interface (e.g they are always provided by
> >> the parent and thus should not be exposed to end users) and some which are
> >> extra (and should always be provided, or at least exposed to end users) you
> >> have no way to differentiate them.
> >
> > Heat has a way to differentiate them surely, because it has the templates?
> > If you implement it that sounds much more reliable than asking template
> > authors to annotate this stuff manually (for a start the same nested
> > template can be instantiated in different ways, so it's not even possible in
> > general to annotate it in such a way as to indicate which parameters will be
> > set by the parent and which should be set by modifying parameter defaults).
>
> The problem is that many (most?) intrinsic functions return None at
> validation time, which we can't easily distinguish from no value being
> passed into the nested stack.
>
> I'm not saying this can't be fixed, I'm just saying it's probably pretty
> hard from an implementation perspective (Jay and I both had a try at fixing
> this but I'd welcome some fresh eyes on it!).
>
> Some more context here (and Jay mentions a newton targetted bug although
> I'm currently failing to find it):
>
> https://bugs.launchpad.net/heat/+bug/1508857
>
> It's true that no general pattern for annotation is possible, but given a
> sufficiently constrained interface (such as we support in some parts of
> TripleO) it would be a workable alternative to the status-quo (if the
> template annotation was possible, but currently it is not).
>
> >> Example of this here:
> >>
> >> https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/heat-api.yaml#L7
> >>
> >> ServiceNetMap, DefaultPasswords and EndpointMap are internal interfaces,
> >> and the remainder are options for the service that should be exposed to the
> >> user.
> >>
> >> One option would be to add internal interfaces to a parameter_group
> >> "internal", but that then means we can never categorize a parameter as
> >> anything else (such as "deprecated" for example, see below).
> >>
> >> 2. When you ship a template containing parameters, it's impossible to ever
> >> deprecate the parameter names
> >>
> >> The problem here is twofold:
> >>  - We don't provide any means to tag a parameter as deprecated (so that,
> >>    for example, a UI or CLI tool could output an annoying warning to
> >>     encourage not using it)
> >>  - There's no way to map an old (deprecated) name to a new one unless you
> >>    do hacks inside the template such as overwriting one parameter with
> >>    another via str_replace (which won't work for all parameter types, and
> >>    you still can't ever remove the old parameter because there's no channel
> >>    to warn users)
> >>
> >> So, one option here is to add a parameter_group called "deprecated" that's
> >> then introspected by the client during the validation phase, and outputs a
> >> warning when deprecated parameters are used.
> >
> > This seems like a very general problem. I would definitely be in favour of
> > some native support for this in the HOT format.
>
> Sure, so would I - but I remember discussing this in Vancouver, so it'd be
> good to figure out some incremental steps to improve this, again a template
> annotation (parameter_groups or something else) provides a simple interim
> workaround for this non-trivial and long-standing feature gap.
>
> >> 3. No way to subcategorize more than once
> >>
> >> The assumption in the current parameter_group interface is that a UI will
> >> always be built on the assumption that parameters should only ever be in
> >> one group, which may be true in Horizon, but it's not the only Ux design
> >> pattern.
> >
> > I guess that's true if you're willing to tweak the layout a bit:
> >
> > http://i1.wp.com/www.jeffreythompson.org/blog/wp-content/uploads/2012/07/7-WayVennDiagram-web.png
>
> Haha :)
>
> Now imagine a tree view Ux browsing nested parameters_schema for 100+
> openstack services.  You don't always want to group parameters based on the
> 100+ nested templates, so you need some way to alternately filter them.

This is precisely our use case.

>
> > (To be fair a carousel-type UI could actually work quite sanely, with a
> > parameter in multiple groups just showing up on multiple different panels.
> > But shouldn't we be going for lowest common denominator here? i.e. shouldn't
> > the output of validate_template be guaranteed to be usable in all of the UIs
> > out there, rather than saying that one UI knows how to display it so
> > everything else has to just deal with it?)
>
> Sure, but the lowest common denominator doesn't necessarily have to be
> Horizon - the heat integration has (respectfully) always been sparse there,
> and in particular support for huge compositions of nested stacks is poor.
>
> With a little more data we can probably enable multiple UI implementations
> (incuding Horizon) to improve, so I think we should think carefully before
> declaring the status-quo as the lowest common denominator and be done with
> it.
>
> >
> >> Particularly when dealing with filtering lots of nested templates (which
> >> all accept parameters which may exist in some subcategory, such as
> >> "network", "passwords", "advanced" etc.  There's no way to subcategorize
> >> parameters in the heat templates, so we're having to wire in hard-coded
> >> translations outside of heat, because tripleo-ui doesn't work the same as
> >> Horizon (it allows you to browse the nested parameters, and there are a lot
> >> so some subcategories are basically needed here, the horizon flat list
> >> approach won't work).
> >>
> >> Any ideas on the most acceptable path forward here would be appreciated -
> >> Randall mentioned enabling per-parameter tags which is certainly an
> >> option, and having some means to handle deprecation would also be very
> >> good, I'm just not sure on the least impactful way to do this.
> >
> > If you made this actually nested, so it output something like:
> >
> >     "ParameterGroups": [
> >         {
> >             "label": "Parameter groups",
> >             "description": "My parameter groups",
> >             "parameters": [
> >                 "param_name-1",
> >                 "param_name-2"
> >             ],
> >             "subgroups": [
> >                 {
> >                     "label": "Deprecated",
> >                     "parameters": [
> >                         "param_name-2"
> >                     ]
> >                 }
> >             ]
> >         }
> >     ]
> >
> > then a UI would always display the best approximation possible at the level
> > of nesting it chooses to implement (including existing UIs that have already
> > 'chosen' no levels of nesting, by virtue of it not being a thing yet).
>
> Interesting, I'd not considered nesting the groups - perhaps that could be
> an option, the template level interfaces sounds like they might get a
> little unweildy but it's definitely worth some experimenting, thanks!
>
> Steve
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>


-- 
Jason E. Rist
Senior Software Engineer
OpenStack User Interfaces
Red Hat, Inc.
openuc: +1.972.707.6408
mobile: +1.720.256.3933
Freenode: jrist
github/twitter: knowncitizen