Open Stack

Hi,

I must apologise as I believe I was the culprit on at least one of these 
problems.

Do we have this process documented somewhere other than email 
specifically for puppet changes?

Kind regards,
Josh

Rackspace Australia

On 7/3/14 4:03 AM, James E. Blair wrote:
> Hi,
>
> We've got a lot of people interested in working on puppet, which is
> great.  I'm hoping that as a result, we'll have a much cleaner and more
> re-usable system.  But at the moment, the primary purpose of what's in
> the openstack-infra/config repo is to run the systems for the OpenStack
> project.  We need to make these changes with a minimum amount of
> disruption.
>
> Our current testing for puppet changes is sub-standard.  We're even
> unable to fully verify that puppet syntax is correct before actually
> running it.  That means that many simple puppet changes are potentially
> breaking for running OpenStack servers.  Our experience has backed this
> up.  I've had to do two emergency reverts of puppet changes this
> morning.
>
> So, until such a time as we've refactored the puppet repo enough to have
> some reasonable separation and safety, as well as a higher standard of
> testing, I'd like to remind everyone of the policy for approving puppet
> changes[1] (after they have had sufficient review):
>
>    * Someone, generally the person authoring the change, should be
>      prepared to monitor systems (using puppetboard) for any anomalous
>      behavior.  If there's a problem, they should take immediate
>      corrective action, either fixing the bug or proposing a revert and
>      collecting immediate approvals.
>
>    * If you are the author, you should self-approve the change and
>      monitor the results yourself.
>
>    * If you are not the author but are nonetheless prepared to monitor
>      the results in their stead, then you can approve it.
>
>    * If you are not the author but can not commit to monitoring the
>      change, then you should not approve the change.  Instead, if the
>      requisite +2 code review votes have been left, you should leave a
>      comment inviting the author to ping a member of infra-core in
>      #openstack-infra when they are prepared to monitor the results of
>      the change.
>
> Generally, people submitting puppet changes are genuinely interested in
> making sure they work and don't break systems.  We're a pretty small
> team, and this lets everyone help out to the best of their abilities.
> Hopefully we'll get to a better place soon and we can blindly approve
> changes with confidence that if they pass tests they will work.  But
> we're not there yet.
>
> [1] I don't mean all config repo changes, just changes to the config
>      repo that could break puppet -- generally changes that are modifying
>      code in one of puppet's languages.
>
> -Jim
>
> _______________________________________________
> OpenStack-Infra mailing list
> OpenStack-Infra at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra