Open Stack

On 26/11/13 22:24, Tim Schnell wrote:
> Use Case #1
> I see valid value in being able to group templates based on a type or

+1, me too.

> keyword. This would allow any client, Horizon or a Template Catalog
> service, to better organize and handle display options for an end-user.

I believe these are separate use cases and deserve to be elaborated as 
such. If one feature can help with both that's great, but we're putting 
the cart before the horse if we jump in and implement the feature 
without knowing why.

Let's consider first a catalog of operator-provided templates as 
proposed (IIUC) by Keith. It seems clear to me in that instance the 
keywords are a property of the template's position in the catalog, and 
not of the template itself.

Horizon is a slightly different story. Do we even allow people to upload 
a bunch of templates and store them in Horizon? If not then there 
doesn't seem much point in this feature for current Horizon users. (And 
if we do, which would surprise me greatly, then the proposed 
implementation doesn't seem that practical - would we want to retrieve 
and parse every template to get the keyword?)

In the longer term, there seems to be a lot of demand for some sort of 
template catalog service, like Glance for templates. (I disagree with 
Clint that it should actually _be_ Glance the project as we know it, for 
the reasons Steve B mentioned earlier, but the concept is right.) And 
this brings us back to a very similar situation to the operator-provided 
template catalog (indeed, that use case would likely be subsumed by this 
one).

> I believe that Ladislav initially proposed a solution that will work here.
> So I will second a proposal that we add a new top-level field to the HOT
> specification called "keywords" that contains this template type.
>
> 	keywords: wordpress, mysql, etcŠ

+1. If we decide that the template is the proper place for these tags 
then this is the perfect way to do it IMO (assuming that it's actually a 
list, not a comma-separated string). It's a standard format that we can 
document and any tool can recognise, the name "keywords" describes 
exactly what it does and there's no confusion with "tags" in Nova and EC2.

> Use Case #2
> The template author should also be able to explicitly define a help string
> that is distinct and separate from the description of an individual

This is not a use case, it's a specification. There seems to be a lot of 
confusion about the difference, so let me sum it up:

Why - Use Case
What - Specification
How - Design Document (i.e. Code)

I know this all sounds very top-down, and believe me that's not my 
philosophy. But design is essentially a global optimisation problem - we 
need to see the whole picture to properly evaluate any given design (or, 
indeed, to find an alternate design), and you're only giving us one 
small piece near the very bottom.

A use case is something that a user of Heat needs to do.

An example of a use case would be: The user needs to see two types of 
information in Horizon that are styled differently/shown at different 
times/other (please specify) so that they can ______________________.

I'm confident that a valid use case _does_ actually exist here, but you 
haven't described it yet.

> parameter. An example where this use case originated was with Nova
> Keypairs. The description of a keypair parameter might be something like,
> "This is the name of a nova key pair that will be used to ssh to the
> compute instance." A help string for this same parameter would be, "To
> learn more about nova keypairs click on this help article."

It's not at all clear to me that these are different pieces of 
information. They both describe the parameter and they're both there to 
help the user. It would be easier to figure out what the right thing 
would be if you gave an example of what you had in mind for how Horizon 
should display these. Even without that, though, it seems to me that the 
help is just adding more detail to the description.

One idea I suggested in the review comments is to just interpret the 
first paragraph as the description and any subsequent paragraphs as the 
help. There is ample precedent for that kind of interpretation in things 
like Python docstrings and Git commit messages.

> I propose adding an additional field to the parameter definition:
> 	
> 	Parameters:
> 		<parameter name>:
> 			description: This is the name of a nova key pair that will be used to
> ssh to the compute instance.
> 			help: To learn more about nova key pairs click on this <a
> href="/some/url/">help article</a>.

(Side note: you're seriously going to let users stick HTML in the 
template and then have the dashboard display it?  Yikes.)

> Use Case #3
> Grouping parameters would help the client make smarter decisions about how
> to display the parameters for input to the end-user. This is so that all
> parameters related to some database resource can be intelligently grouped

+1 sounds reasonable

> together. In addition to grouping these parameters together, there should
> be a method to ensuring that the order within the group of parameters can
> be explicitly stated. This way, the client can return a group of database

Veering into specification territory again.

> parameters and the template author can indicate that the database instance
> name should be first, then the username, then the password, instead of
> that group being returned in a random order.

+2 random order sucks

>
> 	Parameters:
> 		db_name:
> 			group: db
> 			order: 0
> 		db_username:
> 			group: db
> 			order: 1
> 		db_password:
> 			group: db
> 			order: 2
> 		web_node_name:
> 			group: web_node
> 			order: 0
> 		keypair:
> 			group: web_node
> 			order: 1

-2 this is horrible.

Imagine how much work this is for the poor author! At least they don't 
have to maintain parallel hierarchies of matching key names like in the 
original proposal, but they still have to manually maintain multiple 
lists of orderings. What if you wanted to add another parameter at the 
beginning? Maybe we should encourage authors to number parameters with 
multiples of 10. Like BASIC programmers in the '80s.

And of course if you don't specify the order explicitly then you get 
random order again. Sigh.

There's only one way that this is even remotely maintainable for a 
template author, and that's if they group and order stuff manually 
anyway (like you have in your example - people will do this 
automatically by themselves even if the syntax doesn't require them to). 
Since they have to do this, just display the parameters in the UI in the 
same order that they are defined in the file. This does the Right Thing 
even if the author doesn't know about it, unlike the explicit order 
thing which completely breaks down if the order is not explicitly 
stated. You probably won't even have to document it because literally 
100% of people will either (a) not care, or (b) expect it to work that 
way anyway. In fact, you will almost certainly get bug reports if you 
don't display them in the same order as written.

To prove that this is not difficult I even implemented the code for you: 
https://review.openstack.org/#/c/56703/7/doc/source/template_guide/hot_spec.rst 
(in the comments).


Grouping could be easily accomplished with a simple naming convention. e.g.

   parameters:
       db:name:
           ...
       db:username:
           ...
       db:password:
           ...
       web_node:node_name:
           ...
       web_node:keypair:
           ...

I can write you the code for grouping these too:

   groups = itertools.groupby(template['parameters'],
                              lambda k: (k.split(':', 1)[:-1] or
                                         [None])[0])

This method is unambiguous (there's no way to e.g. put a parameter into 
multiple groups), gives a pretty passable result even on a front-end 
that doesn't support it and just sorts by name, is simple to document, 
requires no changes to Heat and is trivial to implement in the front-end.

> These are the use cases that have been clearly defined. The original

Specifications.

> purpose of the metadata section was to "future-proof" (I say future-proof,
> you say pre-optimize ;) ) rapid iterations to potential client design. The
> intent of the strategy was so that we did not overburden the Heat Core
> team with requirements that may fluctuate or change as we attempt to
> improve the user experience of Heat for the community. In hindsight I can
> see how that intent was misconstrued and may have come off as
> condescending and I apologize for that.

Nobody is more invested in having good front-end tools for Heat than the 
core team. So we're naturally concerned by the prospect of front-ends 
implementing a bunch of ideas that affect our mutual users and - 
candidly - range in quality from 'meh' to really very bad. And the 
prospect of opening up the template format to multiple groups 
independently and rapidly implementing (possibly bad) ideas without the 
need to even ask for our input - which is to say, your actual stated 
intent - is genuinely frightening.

Heat Core possesses a wealth of knowledge about how Heat works and how 
it interacts with the world. You should make use of this knowledge by 
talking to us *before* you decide what the answer is. You'll find us in 
all the usual public forums, because this is an Open Source project. If 
you are optimising across a global set of design constraints that 
includes Heat, then you need to either do it in those same forums or be 
prepared to reproduce the relevant parts of the process in those forums 
(the latter is very hard work, as you're discovering) if you want to 
make changes in Heat as a result, and it would be highly advantageous to 
you, your users and us (in that order) if you did so even when you don't 
feel the need to make resulting changes in Heat.

> I think that these proposed use cases will add real value to Heat and
> would require only changes to the HOT specification. If these are
> acceptable please let me know and I will drop the existing blueprint and
> patches in favor of something that implements this design. If there are
> issues to discuss here please suggest alternative solutions so that we can
> continue to move forward with these improvements.

There are many issues to discuss, but each of these is already strictly 
better than its correspondent proposed in the existing blueprint 
(update-stack-metadata-spec) IMO.

> I have no doubt that this is just the beginning of plenty of requirements
> being driven from an attempt to improve the overall end user experience of
> Heat and I will continue to vet the use cases and proposed implementations
> with the community before suggesting a solution. I know that this proposal
> leaves out a few use cases that have been discussed like template
> author/version and that is because I believe that some of the arguments
> made are valid and should be looked into before proposing changes to Heat.
> Most of those other use cases probably can be served from some other
> solution like git or the future template catalog.

Sounds good, thanks.

cheers,
Zane.