Open Stack

Hi all,

I guess there are two issues here: one is the definition of the model
(the parameters, tags or whatever that you want to track), and the other
one is to which extent it can be computed automatically, or needs human
intervention (or human interpretation).

For the first one, in other cases I've successfully used GQM
(goal-question-metric) [1] [2]

[1] http://en.wikipedia.org/wiki/GQM
[2]
http://drum.lib.umd.edu/bitstream/1903/7538/1/Goal_Question_Metric.pdf

For those of you unfamiliar with GQM, the "goal(s)" are like the main
objetives sought. In the etherpad I see a bit of this, but not much. I
see a lot of the next step, the "questions". Maybe grouping them by
goal, and clearly stating the goal would help to structure questions a
bit. In this case, it would be a matter of agreeing on which goals do
the "downstream users" would have for the tags, and then refining them
in questions (by grouping / reformulating the current ones). Once this
is done, specific metrics are defined for each question. The process is
iterative: after having the first list of metrics, usually you have to
come back to goals, because some are not well defined, or no useful
metrics were found, or some important goals are found to be mising.

For example, one goal could be: "learn about how easy to install the
module is". From this, I see some questions in the current version of
the etherpad:

      * Is it packaged?
      * Does it include documentation about installation?

But questions are a bit interlaced with goals. For example, "Dependency
between projects" is probably a goal, "Learning about dependencies of a
given module", with questions that could be:

  * Do the module have dependencies defined?
  * Do defined dependencies have versions specified?
  * Are defined dependencies tested for the release XX of the module?
  * How recent are dependencies of the module?
  * How many dependencies has the module? (this could be recursive)

Metrics for this one could be defined such as "Existence of formal
dependencies file" (boolean), "number of dependencies" (int),
"specification of versions for dependencies" (float, fraction of
dependencies with version), "median age of dependencies" (int, days
since release of each version of each dependency, for example), etc.

If you agree to give GQM a try, I could state the goals I see in the
current Etherpad, and the questions for them, with any of you who see
this as interesting.

[Brief note about me: I've participated in the Polarsys Maturity Model,
and in other evaluation models for software projects in the past, and
I'm also involved in the production of the data for the dashboard at
http://activity.openstack.org.]

Saludos,

	Jesus.

On Tue, 2015-05-05 at 11:31 -0700, Stefano Maffulli wrote:
> Hello folks,
> 
> in PHL meeting the discussion on project tags[1] highlighted that
> operators are looking for answers to questions like:
> 
>    * Are fixes backported? Or Consumes Master to get fixes?
>    * Is it packaged?
>    * Is it stable? Mature?
>    * is it tested in Tempest/Rally/whatever so I can test before I
> deploy changes?
>    * What API Version does it offer
>    * Is documentation available
> 
> I think these questions come down to identifying a 'maturity model' or
> 'evaluation model' that can help people select OpenStack software to
> solve their needs.  The good news is that other communities have been in
> similar situation and looking at them can provide a path for us to
> follow (and mistakes to avoid).
> 
> Apache and Polarsys (an Eclipse working group) have each developed a
> maturity model for their own purposes[2][3]. Each one is focused on
> relevant aspects to the respective community, but all are similar in how
> they are produced. One part of the model comes directly from metrics
> retrieved from repositories (such as this diversity metric, which we
> track with the diverse-affiliation tag[4]), a part needs human
> intervention (such as the release:managed or release:has-stable-branches
> which we add semi-manually [4]).
> 
> I'd like to continue the conversation about an OpenStack Maturity Model
> that can be consumed by users downstream. Any suggestions on how to move
> this project forward?
> 
> /stef
> 
> 
> [1] https://etherpad.openstack.org/p/PHL-ops-tags
> [2]
> https://community.apache.org/apache-way/apache-project-maturity-model.html
> [3] http://dashboard.castalia.camp/documentation/ and
> https://polarsys.org/wiki/Maturity_Assessment_WG
> [4] https://review.openstack.org/#/c/179258/ and
> http://governance.openstack.org/reference/tags/team_diverse-affiliatio
> 
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

-- 
Bitergia: http://bitergia.com
/me at Twitter: https://twitter.com/jgbarah