Open Stack

> On 2016-05-10 19:47, Doug Hellmann wrote:
> > Sorry, I missed the reply-all button.
> >
> > Doug
> >
> > Excerpts from Doug Hellmann's message of 2016-05-10 13:45:19 -0400:
> >> Excerpts from Andreas Jaeger's message of 2016-05-10 19:03:12 +0200:
> >>> On 05/10/2016 06:36 PM, Ronald Bradford wrote:
> >>>> Andreas,
> >>>>
> >>>> I would advocate for NOT moving flagmappings files to individual project
> >>>> repositories.   Having worked out recently how configuration tables are
> >>>> generated and was able to generate them myself I considered how to
> >>>> improve the process in relation to using the existing oslo.config
> >>>> generation tools.
> >>>
> >>> the idea here is to move this to the creation of the options. So,
> >>> instead of a flagmappings file, it would be part of each configuration -
> >>> and thus whenever a new change gets added, the config table value would
> >>> be added as well.
> >>>
> >>>>
> >>>> The primary reason for not moving the flagmappings file is you are then
> >>>> dependent on the projects review cycle (and processes) to get these
> >>>> changes approved.  This includes the load on the gate, and applicable +2
> >>>> by cores (which on major projects have their own priorities for
> >>>> features) etc.  I would not like to see documentation generation data
> >>>> dependent on all applicable projects.
> >>>
> >>> I agree - that's why it should be change itself that adds these.
> >>>
> >>>> I see the bigger issue is the current process for generating the config
> >>>> guides including the frequency and accuracy during the cycle.
> >>>>
> >>>> In a nutshell, it's a huge pull process, where at some time somebody
> >>>> runs the tools which pulls in all projects, and dependencies and uses a
> >>>> doc specific version to identify all the config options, merge with
> >>>> flagmappings and produce configuration tables.
> >>>
> >>> And the way forward would be to have oslo.config generate this
> >>> automatically for us.
> >>>
> >>> Let's see what Doug and KATO come up with.
> >>>
> >>>> Ideally, a more optimal means is to create a push process, that is, if
> >>>> any configuration settings changes within a project during a commit,
> >>>> this data change is pushed, so that the documentation can be updated
> >>>> accordingly when applicable.  Nice in theory, in practice it's more
> >>>> complex. Some high level thoughts on the process.
> >>>>
> >>>> Some projects now can generate a sample configuration file (tox
> >>>> -egenconfig).  In Oslo we are hoping to increase this usage, and use the
> >>>
> >>> And we just want to get rid of this process, Doug worked on a way to
> >>> include the sample configs as part of documentation builds.
> >>>
> >>>> developer docs as a place to store these files within a project repo.
> >>>> Leveraging this existing functionality, if we could detect a change in
> >>>> the sample configuration file during a review commit, then we have the
> >>>> notification process (keystone already has a bot that does this) that is
> >>>> the basic information trigger of a new change needed in docs for config.
> >>>> Now, what is that change of configuration is more complex, and it would
> >>>> seem a logical necessity to use a neutral format (e.g. yaml) to record
> >>>> the parsed configuration options, and enable this format to be saved
> >>>> somewhere/somehow.  This becomes both the input to the oslo.config
> >>>> generator (i.e. we effectively take current functionality and split it
> >>>> into two parts, the parsing of configuration options into yaml format,
> >>>> and the generation of sample config files, and developer docs sphinx
> >>>> extensions using the yaml format. The yaml data also forms the basis of
> >>>> data for the configuration guide.
> >>>> One may argue why break up the oslo.config work because it's working,
> >>>> well because the inputs are used to generate multiple forms of
> >>>> documentation and at present only some forms are being used.
> >>>>
> >>>> I am certainly not familiar with the infra work for bots, and how within
> >>>> a gate to create, propose and commit work of a produced yaml file, or
> >>>> change in yaml file.
> >>>>
> >>>> It does seem like this could be a lot of work, but it's the foundation
> >>>> of converging the work the documentation team does with tools that exist
> >>>> for the development side, as well as reducing the tool complexity that
> >>>> exists in doc tools now.
> >>>
> >>>
> >>> thanks for the comments. Let's see what Doug and KATO will work out.
> >>>
> >>> I would rather use oslo.config to generate RST tables to include than
> >>> the current process. Using oslo.config the same way that projects run
> >>> tox -egenconfig today, I hope that Config Ref generation becomes easier,
> >>>
> >>> Andreas
> >>
> >> One of the unfortunate thing about the summits is that although we can
> >> have lots of face to face conversations, many of them happen in
> >> parallel and so we're not all present for them. At least 3 separate
> >> sessions had parts of conversations that related to this, for example.
> >>
> >> Based on my randomly serendipitous participation in one of the infra
> >> sessions [1], for example, I think we may not want a bot proposing changes
> >> to the docs repo any more, because infra is trying to cut back on the
> >> number of those because each one requires access to the CI systems with
> >> credentials on them.
> >>
> >> The design requirements for producing the config reference, as I
> >> understand them are:
> >>
> >> 1. Someone building the config reference should not need to check out
> >>    all of the source for OpenStack and install it in order for the build
> >>    to work.
> >>
> >>
> >> 2. The config reference maintainers should not have to ever manually
> >>    "pull" the config info from other projects (as is being done now).
> 
> 
> Both are great if we can achieve them - but if there is some scripting -
> like now - that does it, it's fine for me.
> 
> My goals are more basic:
> 
> * Minimal scripting on docs side, we should not reinvent the wheel.
> There should be no need to fix our scripting with every release since
> projects make changes or oslo.config improves...

+1

> * Make it easy to generate list of new options, changed options, removed
> options.

+1


> * Remove any manual steps like updating flagmappings from the process.
> The projects know far better which options belong to each other...

+1

> 
> What do others think?
> 
> Andreas
> 
> >> 3. It should be possible to build the config reference offline after
> >>    downloading all of its parts (this supports the distro build case
> >>    where they don't like the build grabbing files from the internet
> >>    during the build).

+1

> >> 4. Updates to the projects included in the config reference should be
> >>    made automatically when configuration-related changes merge in
> >>    projects.

+1

> >> 5. No proposal bot.

+1
This bot may be needed for No.4 implementation,
but it is better no proposal bot.

Thanks,
KATO Tomoyuki


> >> I have two approaches to suggest, there may be more.
> >>
> >> A. Single guide
> >>
> >>    Add a step to the post-merge queue for projects, to be run after a
> >>    patch is approved and is merged into the repo, to build a data file
> >>    and publish it to a static site. The data file contents are TBD, but
> >>    would at the very least include all of the information we can extract
> >>    about a configuration option, and would probably need to include the
> >>    values currently stored in the flagmappings file.
> >>
> >>    Update the config reference guide build to pull the data file(s) from
> >>    that static site and use them to build the reference. This needs to
> >>    happen in a way that an average person running the build from source
> >>    doesn't have to know to download the files separately, but the distro
> >>    build jobs *can* download the files separately and disable the
> >>    download in the build process.
> >>
> >>    Add another step to the post-merge queue to trigger the config
> >>    reference build job, so that after patches merge in a project the
> >>    guide is automatically updated. We would end up building the guide a
> >>    lot, potentially when nothing changed at all, but we can optimize
> >>    that case away later.
> >>
> >> B. Separate guide per project.
> >>
> >>    Do away with the current single guide in favor of a collection of
> >>    project-specific guides, similar to what we do with release notes and
> >>    API references.
> >>
> >>    Manage the content of the guide in the same git repo as the project
> >>    source, and trigger a build when something merges (again, just like
> >>    with release notes).
> >>
> >>    Publish a list of the guides from some central repo, and the content
> >>    of each individual guide from the project repo hosting it.
> >>
> >> Option A has the benefit of centralizing the configuration info, as we
> >> do now. It is a bit more complex to implement, but we have a similar
> >> pattern for a lot of cases so it shouldn't be too bad. The changes to
> >> oslo.config to produce the data file should be straightforward, although
> >> we still need to solve the flagmappings question.
> >>
> >> Option B has the benefit of simplicity (we have a lot of similar doc
> >> build jobs already) at the expense of spreading the work out into a lot
> >> of repositories. OTOH, that expense also distributes the contribution
> >> load to the teams that own the code, so maybe that's a benefit not
> >> a drawback.  It would mean potentially duplicating some information,
> >> like the introduction and docs for library options, but it places all of
> >> the information relevant to configuring a given project in one location,
> >> which may be beneficial to readers.  It would eliminate (or reduce?) the
> >> need for flagmappings because the existing oslo.config sphinx extension
> >> can manage sets of options coming from libraries, and can also be made
> >> to manage oslo.config option groups, in a way that gives basically the
> >> same features (at least as I understand the way flagmappings is used
> >> now). It also has the benefit of giving project teams a place to put
> >> narrative configuration instructions, like what the glance team has in
> >> their developer documentation already.
> >>
> >> I have, so far, been thinking only of ways to implement Option A, but
> >> after the success of the install guide session at the summit (where we
> >> agreed to encourage having multiple installation guides for different
> >> purposes/audiences), I'm now actually leaning in support of Option B.
> >>
> >> I'm curious to know what other folks think, though, and I'll be happy to
> >> try to help implement either option.
> 
> 
> 
> --
>  Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
>   SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
>    GF: Felix Imendörffer, Jane Smithard, Graham Norton,
>        HRB 21284 (AG Nürnberg)
>     GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126
> 
> 
> _______________________________________________
> OpenStack-docs mailing list
> OpenStack-docs at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-docs