Open Stack

----- Original Message -----
> From: "Gauvain Pocentek" <gauvain.pocentek at objectif-libre.com>
> To: "Anne Gentle" <annegentle at justwriteclick.com>
> Cc: "openstack-docs" <openstack-docs at lists.openstack.org>
> Sent: Saturday, November 16, 2013 12:42:36 AM
> Subject: Re: [Openstack-docs] Broken MySQL chapter
> 
> Hey,
> 
> Le 15/11/2013 22:01, Anne Gentle a écrit :
> > Any root cause we can avoid?
> 
> If we were dealing with code I'd say "we lack unit tests", but for
> documentation it is probably very hard to manage (are there some tools
> out there to test documentation?).

Doing "proper" CI with full coverage that actually extracts and runs all the instructions is difficult, but in this case I think we could pick the cause of the issue up in validation. I've proposed an update to test.py here:

    https://review.openstack.org/#/c/56790/

For the change in question the output would have been:

python ./tools/test.py --check-syntax
Checking syntax of xml files...
 Validating install-guide/ch_basics.xml
  install-guide/ch_basics.xml: {http://docbook.org/ns/docbook}itemizedlist os profiling (['opensuse', 'sles']) conflicts with os profiling of {http://docbook.org/ns/docbook}screen on line 276.
Check failed, validated 1 xml files with 1 failures.
    
> My feeling is that the bug itself is not a problem (a "broken" trunk is
> not that big a deal) but it shouldn't have landed in the stable branch.
> Maybe the commit that caused the problem was too large for a backport,
> and/or the backport happened too quickly after the merge in master?
> 
> Maybe we should review how/when the backports are applied? What do you
> think?

For both the initial commit and the backport this had to go through both the validation checks and manual review. As a result the only way I see to avoid this is more checks (as above) and more thorough reviews (easier said than done sometimes!).

I don't see additional red tape around backports as necessarily being able to prevent issues like this where there isn't a check in the validation to pick it up and human reviewers didn't notice.

Steve