Open Stack

Excerpts from Tim Schnell's message of 2013-11-26 13:24:22 -0800:
> So the originally question that I attempted to pose was, "Can we add a
> schema-less metadata section to the template that can be used for a
> variety of purposes?". It looks like the answer is no, we need to discuss
> the features that would go in the metadata section and add them to the HOT
> specification if they are viable. I don't necessarily agree with this
> answer but I accept it as viable and take responsibility for the
> long-winded process that it took to get to this point.
> 
> I think some valid points have been made and I have re-focused my efforts
> into the following proposed solution.
> 
> I am fine with getting rid of the concept of a schema-less metadata
> section. If we can arrive at a workable design for a few use cases then I
> think that we won't need to discuss any of the options that Zane mentioned
> for handling the metadata section, comments, separate file, or in the
> template body.
> 
> Use Case #1
> I see valid value in being able to group templates based on a type or
> keyword. This would allow any client, Horizon or a Template Catalog
> service, to better organize and handle display options for an end-user.
> 
> I believe that Ladislav initially proposed a solution that will work here.
> So I will second a proposal that we add a new top-level field to the HOT
> specification called "keywords" that contains this template type.
> 
>     keywords: wordpress, mysql, etcŠ
> 

What is the "use case" here. For me, a use case needs to be specific to
be useful at helping to guide more generic feature design. What I see
above is a generic feature specification with no actual concrete need
for it. These keywords should be in the template description, which
should be full-text searchable anyway. "Group templates based on a type
or keyword." could mean a lot of things to a lot of different classes
of Heat users.

> 
> Use Case #2
> The template author should also be able to explicitly define a help string
> that is distinct and separate from the description of an individual
> parameter. An example where this use case originated was with Nova
> Keypairs. The description of a keypair parameter might be something like,
> "This is the name of a nova key pair that will be used to ssh to the
> compute instance." A help string for this same parameter would be, "To
> learn more about nova keypairs click on this help article."
> 
> I propose adding an additional field to the parameter definition:
>     
>     Parameters:
>         <parameter name>:
>             description: This is the name of a nova key pair that will be used to
> ssh to the compute instance.
>             help: To learn more about nova key pairs click on this <a
> href="/some/url/">help article</a>.
>

+1. A help string per parameter is a fantastic idea. Sounds like a nice
analog to doc-strings.

> Use Case #3
> Grouping parameters would help the client make smarter decisions about how
> to display the parameters for input to the end-user. This is so that all
> parameters related to some database resource can be intelligently grouped
> together. In addition to grouping these parameters together, there should
> be a method to ensuring that the order within the group of parameters can
> be explicitly stated. This way, the client can return a group of database
> parameters and the template author can indicate that the database instance
> name should be first, then the username, then the password, instead of
> that group being returned in a random order.
> 
>     Parameters:
>         db_name:
>             group: db
>             order: 0
>         db_username:
>             group: db
>             order: 1
>         db_password:
>             group: db
>             order: 2
>         web_node_name:
>             group: web_node
>             order: 0
>         keypair:
>             group: web_node
>             order: 1
>

+1 for grouping. Your use case is perfectly specified above. Love it.

However, your format feels very forced. Integers for ordering? We have
lists for that.

How about this:

parameters:
  db:
    type: group
    parameters:
      - db_name
      - db_username
      - db_password
  web_node:
    type: group
    parameters:
      - web_node_name
      - keypair
  db_name:
    type: string
  ...


This way we are just providing the groupings outside of the parameters
themselves. Using a list means the order is as it appears here.

Another option is to allow specifying whole parameters in-line there,
like

parameters:
  db:
    type: group
    parameters:
      - db_name:
        type: string

But that opens up a can of worms that we can leave closed for now and
just do the above.

> These are the use cases that have been clearly defined. The original
> purpose of the metadata section was to "future-proof" (I say future-proof,
> you say pre-optimize ;) ) rapid iterations to potential client design. The
> intent of the strategy was so that we did not overburden the Heat Core
> team with requirements that may fluctuate or change as we attempt to
> improve the user experience of Heat for the community. In hindsight I can
> see how that intent was misconstrued and may have come off as
> condescending and I apologize for that.
> 

Why would you be worried about overburdening _a team you can and should
join_? Heat core isn't for special people. It is for involved people.
You are going to be involved.. so.. join us.

Also _all_ of the requirements on Heat fluctuate wildly. That's what we're
here for. That's what makes Open Source amazing. You can get your wildly
fluctuating stuff _in to the product_.  Just participate for real please!

> I think that these proposed use cases will add real value to Heat and
> would require only changes to the HOT specification. If these are
> acceptable please let me know and I will drop the existing blueprint and
> patches in favor of something that implements this design. If there are
> issues to discuss here please suggest alternative solutions so that we can
> continue to move forward with these improvements.
> 

I think you're right for the one's you've been clear about specifying.
These do add value. And perhaps the others do too. But what is weird
for me is that they're just showing up in reviews.. not on the ML first.

> I have no doubt that this is just the beginning of plenty of requirements
> being driven from an attempt to improve the overall end user experience of
> Heat and I will continue to vet the use cases and proposed implementations
> with the community before suggesting a solution. I know that this proposal
> leaves out a few use cases that have been discussed like template
> author/version and that is because I believe that some of the arguments
> made are valid and should be looked into before proposing changes to Heat.
> Most of those other use cases probably can be served from some other
> solution like git or the future template catalog.

As Zane suggested.. just stuffing things in git would go a long way to
making template metadata such as author/version/etc. available, and would
also cover efficient distribution, even security (via signed tags). Why
not follow the example OpenStack has set with python packaging and pbr,
where things relating to VCS are just pulled from vcs rather than being
maintained in the wrong place (in code).

More and more I think that we need to think of a distribution format,
and that format may well be based on git trees. If we put it all in the
template, we will just make inaccurate templates the norm.