Open Stack

On 11/27/13 1:17 PM, "Zane Bitter" <zbitter at redhat.com> wrote:

>On 27/11/13 18:16, Tim Schnell wrote:
>>
>> On 11/27/13 10:09 AM, "Zane Bitter" <zbitter at redhat.com> wrote:
>>
>>> On 26/11/13 22:24, Tim Schnell wrote:
>>>> Use Case #1
>>>> I see valid value in being able to group templates based on a type or
>>>
>>> +1, me too.
>>>
>>>> keyword. This would allow any client, Horizon or a Template Catalog
>>>> service, to better organize and handle display options for an
>>>>end-user.
>>>
>>> I believe these are separate use cases and deserve to be elaborated as
>>> such. If one feature can help with both that's great, but we're putting
>>> the cart before the horse if we jump in and implement the feature
>>> without knowing why.
>>>
>>> Let's consider first a catalog of operator-provided templates as
>>> proposed (IIUC) by Keith. It seems clear to me in that instance the
>>> keywords are a property of the template's position in the catalog, and
>>> not of the template itself.
>>>
>>> Horizon is a slightly different story. Do we even allow people to
>>>upload
>>> a bunch of templates and store them in Horizon? If not then there
>>> doesn't seem much point in this feature for current Horizon users. (And
>>> if we do, which would surprise me greatly, then the proposed
>>> implementation doesn't seem that practical - would we want to retrieve
>>> and parse every template to get the keyword?)
>>
>> Correct, at the moment, Horizon has no concept of a template catalog of
>> any kind.
>>
>> Here is my use case for including this in the template for Horizon:
>> (I'm going to start moving these to the wiki that Steve Baker setup)
>>
>> Let's assume that an end-user of Heat has spun up 20 stacks and has now
>> requested help from a Support Operator of heat. In this case, the
>>end-user
>> did not have a solid naming convention for naming his stacks, they are
>>all
>> named "tim1", "tim2", etcŠ And also his request to the Support Operator
>> was really vague, like "My Wordpress stack is broken."
>>
>> The first thing that the Support Operator would do, would be to pull up
>> end-user's stacks in either Horizon or via the heat client cli. In both
>> cases, at the moment, he would then have to either stack-show on each
>> stack to look at the description of the stack or ask the end-user for a
>> stack-id/stack-name. This currently gets the job done but a better
>> experience would be for stack-list to already display some keywords
>>about
>> each stack so the Support Operator would have to do less digging.
>>
>> In this case the end-user only has one Wordpress stack so he would have
>> been annoyed if the Support Operator requested more information from
>>him.
>> (Or maybe he has more than one wordpress stack, but only one currently
>>in
>> CREATE_FAILED state).
>
>This is a truly excellent example of a use case, btw. Kudos.
>
>> As a team, we have already encountered this exact situation just doing
>> team testing so I imagine that others would find value in a consistent
>>way
>> to determine at least a general purpose of a stack, from the stack-list
>> page. Putting the stack-description in the stack-list table would take
>>up
>> too much room from a design standpoint.
>>
>> Once keywords has been added to the template then part of the blueprint
>> would be to return it with the stack-list information.
>
>Other OpenStack APIs have user-defined 'tags' associated with resources.
>Maybe we should implement something like this for Heat stacks also.
>(i.e. it's in the API, not the template.) When the dashboard launches a
>template out of the template catalog, it could automatically populate
>the tags with the ones from the catalog metadata?

Yeah, I think this is starting to go back to what Angus suggested about
having a tag in the template that is used to associate it back to some
catalog metadata which will work. The only reason I'm pushing for this to
be separate is because then we would be relying on only interfacing with
templates from the catalog for this feature. I think the scope of the
template catalog should be to manage the initial
organization/discoverability of the templates but then once we cross into
what kind of features that we want built in to the stack that is
instantiated, relying on the stack to talk back to the catalog metadata
would exclude this from working for stacks that have been instantiated
from templates that did not come from the catalog.

Do we expect the catalog to be the sole interface for instantiating stacks?

>
>>> In the longer term, there seems to be a lot of demand for some sort of
>>> template catalog service, like Glance for templates. (I disagree with
>>> Clint that it should actually _be_ Glance the project as we know it,
>>>for
>>> the reasons Steve B mentioned earlier, but the concept is right.) And
>>> this brings us back to a very similar situation to the
>>>operator-provided
>>> template catalog (indeed, that use case would likely be subsumed by
>>>this
>>> one).
>>>
>>>> I believe that Ladislav initially proposed a solution that will work
>>>> here.
>>>> So I will second a proposal that we add a new top-level field to the
>>>>HOT
>>>> specification called "keywords" that contains this template type.
>>>>
>>>> 	keywords: wordpress, mysql, etcŠ
>>>
>>> +1. If we decide that the template is the proper place for these tags
>>> then this is the perfect way to do it IMO (assuming that it's actually
>>>a
>>> list, not a comma-separated string). It's a standard format that we can
>>> document and any tool can recognise, the name "keywords" describes
>>> exactly what it does and there's no confusion with "tags" in Nova and
>>>EC2.
>>>
>>>> Use Case #2
>>>> The template author should also be able to explicitly define a help
>>>> string
>>>> that is distinct and separate from the description of an individual
>>>
>>> This is not a use case, it's a specification. There seems to be a lot
>>>of
>>> confusion about the difference, so let me sum it up:
>>>
>>> Why - Use Case
>>> What - Specification
>>> How - Design Document (i.e. Code)
>>>
>>> I know this all sounds very top-down, and believe me that's not my
>>> philosophy. But design is essentially a global optimisation problem -
>>>we
>>> need to see the whole picture to properly evaluate any given design
>>>(or,
>>> indeed, to find an alternate design), and you're only giving us one
>>> small piece near the very bottom.
>>>
>>> A use case is something that a user of Heat needs to do.
>>>
>>> An example of a use case would be: The user needs to see two types of
>>> information in Horizon that are styled differently/shown at different
>>> times/other (please specify) so that they can ______________________.
>>>
>>> I'm confident that a valid use case _does_ actually exist here, but you
>>> haven't described it yet.
>>
>> Here is my use case for separating description and help text:
>>
>> Description and help are separate things from a UI perspective. A
>> description might be displayed as a label in a form or in a paragraph
>> somewhere around the input. A help string is typically displayed as
>>hover
>> text when focusing on the input or hovering/clicking on a question mark
>> icon next to the field. We could technically separate these things in
>>the
>> code but because they serve separate purposes I would prefer to have
>>them
>> be defined explicitly.
>
>Thanks, this is helpful. I'm warming to this; if it's in a separate
>pop-up bubble, as opposed to expanding below the existing description,
>it seems like a separate thing worthy of it's own key.
>
>>>> parameter. An example where this use case originated was with Nova
>>>> Keypairs. The description of a keypair parameter might be something
>>>> like,
>>>> "This is the name of a nova key pair that will be used to ssh to the
>>>> compute instance." A help string for this same parameter would be, "To
>>>> learn more about nova keypairs click on this help article."
>>>
>>> It's not at all clear to me that these are different pieces of
>>> information. They both describe the parameter and they're both there to
>>> help the user. It would be easier to figure out what the right thing
>>> would be if you gave an example of what you had in mind for how Horizon
>>> should display these. Even without that, though, it seems to me that
>>>the
>>> help is just adding more detail to the description.
>>>
>>> One idea I suggested in the review comments is to just interpret the
>>> first paragraph as the description and any subsequent paragraphs as the
>>> help. There is ample precedent for that kind of interpretation in
>>>things
>>> like Python docstrings and Git commit messages.
>>>
>>>> I propose adding an additional field to the parameter definition:
>>>> 	
>>>> 	Parameters:
>>>> 		<parameter name>:
>>>> 			description: This is the name of a nova key pair that will be used
>>>>to
>>>> ssh to the compute instance.
>>>> 			help: To learn more about nova key pairs click on this <a
>>>> href="/some/url/">help article</a>.
>>>
>>> (Side note: you're seriously going to let users stick HTML in the
>>> template and then have the dashboard display it?  Yikes.)
>>
>> FWIW, I said the exact same thing to Keith Bray and his answer was, "why
>> not?"
>
>https://en.wikipedia.org/wiki/Cross-site_scripting
>https://en.wikipedia.org/wiki/Cross-site_request_forgery
>
>Not to mention how easily it can screw up the formatting of the page.
>
>> The UI is already making determinations about what HTML to generate
>>based
>> on the template. For example, the parameter label to display just
>> unslugifies the parameter key. This is a somewhat tangential discussion
>> though, and I do have reservations about it. Maybe Keith can jump in and
>> defend this better.
>
>That's a very different thing to emitting user-generated HTML. Of course
>it is possible to do this safely, but it's a lot of work and very easy
>to get wrong.
>
>It's much easier to recognise (safe) URLs in plain text and convert them
>to links. Or to adopt a de-facto formatting language, like Markdown,
>that is readable in plain text.

I think we are pretty close in agreement on this actually. I said
something very similar in the initial discussion with Keith. I'll re-word
the specification when I move this to the wiki to reflect a plain text url.

>
>>>
>>>> Use Case #3
>>>> Grouping parameters would help the client make smarter decisions about
>>>> how
>>>> to display the parameters for input to the end-user. This is so that
>>>>all
>>>> parameters related to some database resource can be intelligently
>>>> grouped
>>>
>>> +1 sounds reasonable
>>>
>>>> together. In addition to grouping these parameters together, there
>>>> should
>>>> be a method to ensuring that the order within the group of parameters
>>>> can
>>>> be explicitly stated. This way, the client can return a group of
>>>> database
>>>
>>> Veering into specification territory again.
>>>
>>>> parameters and the template author can indicate that the database
>>>> instance
>>>> name should be first, then the username, then the password, instead of
>>>> that group being returned in a random order.
>>>
>>> +2 random order sucks
>>>
>>>>
>>>> 	Parameters:
>>>> 		db_name:
>>>> 			group: db
>>>> 			order: 0
>>>> 		db_username:
>>>> 			group: db
>>>> 			order: 1
>>>> 		db_password:
>>>> 			group: db
>>>> 			order: 2
>>>> 		web_node_name:
>>>> 			group: web_node
>>>> 			order: 0
>>>> 		keypair:
>>>> 			group: web_node
>>>> 			order: 1
>>>
>>> -2 this is horrible.
>>>
>>> Imagine how much work this is for the poor author! At least they don't
>>> have to maintain parallel hierarchies of matching key names like in the
>>> original proposal, but they still have to manually maintain multiple
>>> lists of orderings. What if you wanted to add another parameter at the
>>> beginning? Maybe we should encourage authors to number parameters with
>>> multiples of 10. Like BASIC programmers in the '80s.
>>>
>>> And of course if you don't specify the order explicitly then you get
>>> random order again. Sigh.
>>>
>>> There's only one way that this is even remotely maintainable for a
>>> template author, and that's if they group and order stuff manually
>>> anyway (like you have in your example - people will do this
>>> automatically by themselves even if the syntax doesn't require them
>>>to).
>>> Since they have to do this, just display the parameters in the UI in
>>>the
>>> same order that they are defined in the file. This does the Right Thing
>>> even if the author doesn't know about it, unlike the explicit order
>>> thing which completely breaks down if the order is not explicitly
>>> stated. You probably won't even have to document it because literally
>>> 100% of people will either (a) not care, or (b) expect it to work that
>>> way anyway. In fact, you will almost certainly get bug reports if you
>>> don't display them in the same order as written.
>>>
>>> To prove that this is not difficult I even implemented the code for
>>>you:
>>> 
>>>https://review.openstack.org/#/c/56703/7/doc/source/template_guide/hot_s
>>>pe
>>> c.rst
>>> (in the comments).
>>>
>>>
>>> Grouping could be easily accomplished with a simple naming convention.
>>> e.g.
>>>
>>>    parameters:
>>>        db:name:
>>>            ...
>>>        db:username:
>>>            ...
>>>        db:password:
>>>            ...
>>>        web_node:node_name:
>>>            ...
>>>        web_node:keypair:
>>>            ...
>>>
>>> I can write you the code for grouping these too:
>>>
>>>    groups = itertools.groupby(template['parameters'],
>>>                               lambda k: (k.split(':', 1)[:-1] or
>>>                                          [None])[0])
>>>
>>> This method is unambiguous (there's no way to e.g. put a parameter into
>>> multiple groups), gives a pretty passable result even on a front-end
>>> that doesn't support it and just sorts by name, is simple to document,
>>> requires no changes to Heat and is trivial to implement in the
>>>front-end.
>>
>> You're right my initial proposal is terrible, Steve Baker suggested:
>>
>> parameter-groups:
>> - name: db
>>    description: Database configuration options
>>    parameters: [db_name, db_username, db_password]
>> - name: web_node
>>    description: Web server configuration
>>    parameters: [web_node_name, keypair]
>> parameters:
>>    # as above, but without requiring any order or group attributes
>
>I don't hate this idea, but there are a few downsides to it that I see:
>- Only parameters that are members of a group can be ordered
>- Parameters can be members of multiple groups
>- Authors have to keep two sets of non-co-located data in sync
>- This still does the Wrong Thing by default (i.e. if you don't know
>about it, or you're using a template format [cfn] that doesn't support
>it, or you're using an old template from somewhere)

I agree with these downsides. I think grouping via a parameter naming
convention is definitely cleaner than having to create a new top-level
field. The only subjective response I have is that I'm hesitant to say
that using the naming convention and implicit ordering may not be more
intuitive for the template author.

For example, as a template author I do not understand the purpose of the
naming convention just by looking at an example template. With
parameter-groups, the purpose is much more obvious. By definition of
"parameter-groups" I am defining exactly how I want the parameters
grouped. "db:name" could be doing a bunch of different things, is is
creating the relationship with the resources? What is it doing?

I'm not opposed to it, but it would rely more heavily on documentation.

Thanks,
Tim
>
>> Honestly, I can see it going either way and I do agree with your point
>> about forcing the parameters to only have one group by setting the group
>> in the parameter name. Also, if we agree that Heat should return the
>> parameters already grouped and ordered from the template_validate call
>> then the implementation in the template should be about what we think is
>> easier and more intuitive for the template author.
>
>+1
>
>cheers,
>Zane.
>
>
>_______________________________________________
>OpenStack-dev mailing list
>OpenStack-dev at lists.openstack.org
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev