Open Stack

On 27/11/13 18:16, Tim Schnell wrote:
>
> On 11/27/13 10:09 AM, "Zane Bitter" <zbitter at redhat.com> wrote:
>
>> On 26/11/13 22:24, Tim Schnell wrote:
>>> Use Case #1
>>> I see valid value in being able to group templates based on a type or
>>
>> +1, me too.
>>
>>> keyword. This would allow any client, Horizon or a Template Catalog
>>> service, to better organize and handle display options for an end-user.
>>
>> I believe these are separate use cases and deserve to be elaborated as
>> such. If one feature can help with both that's great, but we're putting
>> the cart before the horse if we jump in and implement the feature
>> without knowing why.
>>
>> Let's consider first a catalog of operator-provided templates as
>> proposed (IIUC) by Keith. It seems clear to me in that instance the
>> keywords are a property of the template's position in the catalog, and
>> not of the template itself.
>>
>> Horizon is a slightly different story. Do we even allow people to upload
>> a bunch of templates and store them in Horizon? If not then there
>> doesn't seem much point in this feature for current Horizon users. (And
>> if we do, which would surprise me greatly, then the proposed
>> implementation doesn't seem that practical - would we want to retrieve
>> and parse every template to get the keyword?)
>
> Correct, at the moment, Horizon has no concept of a template catalog of
> any kind.
>
> Here is my use case for including this in the template for Horizon:
> (I'm going to start moving these to the wiki that Steve Baker setup)
>
> Let's assume that an end-user of Heat has spun up 20 stacks and has now
> requested help from a Support Operator of heat. In this case, the end-user
> did not have a solid naming convention for naming his stacks, they are all
> named "tim1", "tim2", etcŠ And also his request to the Support Operator
> was really vague, like "My Wordpress stack is broken."
>
> The first thing that the Support Operator would do, would be to pull up
> end-user's stacks in either Horizon or via the heat client cli. In both
> cases, at the moment, he would then have to either stack-show on each
> stack to look at the description of the stack or ask the end-user for a
> stack-id/stack-name. This currently gets the job done but a better
> experience would be for stack-list to already display some keywords about
> each stack so the Support Operator would have to do less digging.
>
> In this case the end-user only has one Wordpress stack so he would have
> been annoyed if the Support Operator requested more information from him.
> (Or maybe he has more than one wordpress stack, but only one currently in
> CREATE_FAILED state).

This is a truly excellent example of a use case, btw. Kudos.

> As a team, we have already encountered this exact situation just doing
> team testing so I imagine that others would find value in a consistent way
> to determine at least a general purpose of a stack, from the stack-list
> page. Putting the stack-description in the stack-list table would take up
> too much room from a design standpoint.
>
> Once keywords has been added to the template then part of the blueprint
> would be to return it with the stack-list information.

Other OpenStack APIs have user-defined 'tags' associated with resources. 
Maybe we should implement something like this for Heat stacks also. 
(i.e. it's in the API, not the template.) When the dashboard launches a 
template out of the template catalog, it could automatically populate 
the tags with the ones from the catalog metadata?

>> In the longer term, there seems to be a lot of demand for some sort of
>> template catalog service, like Glance for templates. (I disagree with
>> Clint that it should actually _be_ Glance the project as we know it, for
>> the reasons Steve B mentioned earlier, but the concept is right.) And
>> this brings us back to a very similar situation to the operator-provided
>> template catalog (indeed, that use case would likely be subsumed by this
>> one).
>>
>>> I believe that Ladislav initially proposed a solution that will work
>>> here.
>>> So I will second a proposal that we add a new top-level field to the HOT
>>> specification called "keywords" that contains this template type.
>>>
>>> 	keywords: wordpress, mysql, etcŠ
>>
>> +1. If we decide that the template is the proper place for these tags
>> then this is the perfect way to do it IMO (assuming that it's actually a
>> list, not a comma-separated string). It's a standard format that we can
>> document and any tool can recognise, the name "keywords" describes
>> exactly what it does and there's no confusion with "tags" in Nova and EC2.
>>
>>> Use Case #2
>>> The template author should also be able to explicitly define a help
>>> string
>>> that is distinct and separate from the description of an individual
>>
>> This is not a use case, it's a specification. There seems to be a lot of
>> confusion about the difference, so let me sum it up:
>>
>> Why - Use Case
>> What - Specification
>> How - Design Document (i.e. Code)
>>
>> I know this all sounds very top-down, and believe me that's not my
>> philosophy. But design is essentially a global optimisation problem - we
>> need to see the whole picture to properly evaluate any given design (or,
>> indeed, to find an alternate design), and you're only giving us one
>> small piece near the very bottom.
>>
>> A use case is something that a user of Heat needs to do.
>>
>> An example of a use case would be: The user needs to see two types of
>> information in Horizon that are styled differently/shown at different
>> times/other (please specify) so that they can ______________________.
>>
>> I'm confident that a valid use case _does_ actually exist here, but you
>> haven't described it yet.
>
> Here is my use case for separating description and help text:
>
> Description and help are separate things from a UI perspective. A
> description might be displayed as a label in a form or in a paragraph
> somewhere around the input. A help string is typically displayed as hover
> text when focusing on the input or hovering/clicking on a question mark
> icon next to the field. We could technically separate these things in the
> code but because they serve separate purposes I would prefer to have them
> be defined explicitly.

Thanks, this is helpful. I'm warming to this; if it's in a separate 
pop-up bubble, as opposed to expanding below the existing description, 
it seems like a separate thing worthy of it's own key.

>>> parameter. An example where this use case originated was with Nova
>>> Keypairs. The description of a keypair parameter might be something
>>> like,
>>> "This is the name of a nova key pair that will be used to ssh to the
>>> compute instance." A help string for this same parameter would be, "To
>>> learn more about nova keypairs click on this help article."
>>
>> It's not at all clear to me that these are different pieces of
>> information. They both describe the parameter and they're both there to
>> help the user. It would be easier to figure out what the right thing
>> would be if you gave an example of what you had in mind for how Horizon
>> should display these. Even without that, though, it seems to me that the
>> help is just adding more detail to the description.
>>
>> One idea I suggested in the review comments is to just interpret the
>> first paragraph as the description and any subsequent paragraphs as the
>> help. There is ample precedent for that kind of interpretation in things
>> like Python docstrings and Git commit messages.
>>
>>> I propose adding an additional field to the parameter definition:
>>> 	
>>> 	Parameters:
>>> 		<parameter name>:
>>> 			description: This is the name of a nova key pair that will be used to
>>> ssh to the compute instance.
>>> 			help: To learn more about nova key pairs click on this <a
>>> href="/some/url/">help article</a>.
>>
>> (Side note: you're seriously going to let users stick HTML in the
>> template and then have the dashboard display it?  Yikes.)
>
> FWIW, I said the exact same thing to Keith Bray and his answer was, "why
> not?"

https://en.wikipedia.org/wiki/Cross-site_scripting
https://en.wikipedia.org/wiki/Cross-site_request_forgery

Not to mention how easily it can screw up the formatting of the page.

> The UI is already making determinations about what HTML to generate based
> on the template. For example, the parameter label to display just
> unslugifies the parameter key. This is a somewhat tangential discussion
> though, and I do have reservations about it. Maybe Keith can jump in and
> defend this better.

That's a very different thing to emitting user-generated HTML. Of course 
it is possible to do this safely, but it's a lot of work and very easy 
to get wrong.

It's much easier to recognise (safe) URLs in plain text and convert them 
to links. Or to adopt a de-facto formatting language, like Markdown, 
that is readable in plain text.

>>
>>> Use Case #3
>>> Grouping parameters would help the client make smarter decisions about
>>> how
>>> to display the parameters for input to the end-user. This is so that all
>>> parameters related to some database resource can be intelligently
>>> grouped
>>
>> +1 sounds reasonable
>>
>>> together. In addition to grouping these parameters together, there
>>> should
>>> be a method to ensuring that the order within the group of parameters
>>> can
>>> be explicitly stated. This way, the client can return a group of
>>> database
>>
>> Veering into specification territory again.
>>
>>> parameters and the template author can indicate that the database
>>> instance
>>> name should be first, then the username, then the password, instead of
>>> that group being returned in a random order.
>>
>> +2 random order sucks
>>
>>>
>>> 	Parameters:
>>> 		db_name:
>>> 			group: db
>>> 			order: 0
>>> 		db_username:
>>> 			group: db
>>> 			order: 1
>>> 		db_password:
>>> 			group: db
>>> 			order: 2
>>> 		web_node_name:
>>> 			group: web_node
>>> 			order: 0
>>> 		keypair:
>>> 			group: web_node
>>> 			order: 1
>>
>> -2 this is horrible.
>>
>> Imagine how much work this is for the poor author! At least they don't
>> have to maintain parallel hierarchies of matching key names like in the
>> original proposal, but they still have to manually maintain multiple
>> lists of orderings. What if you wanted to add another parameter at the
>> beginning? Maybe we should encourage authors to number parameters with
>> multiples of 10. Like BASIC programmers in the '80s.
>>
>> And of course if you don't specify the order explicitly then you get
>> random order again. Sigh.
>>
>> There's only one way that this is even remotely maintainable for a
>> template author, and that's if they group and order stuff manually
>> anyway (like you have in your example - people will do this
>> automatically by themselves even if the syntax doesn't require them to).
>> Since they have to do this, just display the parameters in the UI in the
>> same order that they are defined in the file. This does the Right Thing
>> even if the author doesn't know about it, unlike the explicit order
>> thing which completely breaks down if the order is not explicitly
>> stated. You probably won't even have to document it because literally
>> 100% of people will either (a) not care, or (b) expect it to work that
>> way anyway. In fact, you will almost certainly get bug reports if you
>> don't display them in the same order as written.
>>
>> To prove that this is not difficult I even implemented the code for you:
>> https://review.openstack.org/#/c/56703/7/doc/source/template_guide/hot_spe
>> c.rst
>> (in the comments).
>>
>>
>> Grouping could be easily accomplished with a simple naming convention.
>> e.g.
>>
>>    parameters:
>>        db:name:
>>            ...
>>        db:username:
>>            ...
>>        db:password:
>>            ...
>>        web_node:node_name:
>>            ...
>>        web_node:keypair:
>>            ...
>>
>> I can write you the code for grouping these too:
>>
>>    groups = itertools.groupby(template['parameters'],
>>                               lambda k: (k.split(':', 1)[:-1] or
>>                                          [None])[0])
>>
>> This method is unambiguous (there's no way to e.g. put a parameter into
>> multiple groups), gives a pretty passable result even on a front-end
>> that doesn't support it and just sorts by name, is simple to document,
>> requires no changes to Heat and is trivial to implement in the front-end.
>
> You're right my initial proposal is terrible, Steve Baker suggested:
>
> parameter-groups:
> - name: db
>    description: Database configuration options
>    parameters: [db_name, db_username, db_password]
> - name: web_node
>    description: Web server configuration
>    parameters: [web_node_name, keypair]
> parameters:
>    # as above, but without requiring any order or group attributes

I don't hate this idea, but there are a few downsides to it that I see:
- Only parameters that are members of a group can be ordered
- Parameters can be members of multiple groups
- Authors have to keep two sets of non-co-located data in sync
- This still does the Wrong Thing by default (i.e. if you don't know 
about it, or you're using a template format [cfn] that doesn't support 
it, or you're using an old template from somewhere)

> Honestly, I can see it going either way and I do agree with your point
> about forcing the parameters to only have one group by setting the group
> in the parameter name. Also, if we agree that Heat should return the
> parameters already grouped and ordered from the template_validate call
> then the implementation in the template should be about what we think is
> easier and more intuitive for the template author.

+1

cheers,
Zane.