Open Stack

Because this has come up a few times during freeze, it's probably worth
actively advertising the policy we've developed on dropping / skipping a
Tempest test.

Tempest tests encode the behavior of the system (to some degree), which
means that once we know the behavior of the system, if code in a core
project can only land if we skip or drop a tempest test, that's clearly
a behavior change.

We want to be really deliberate about this, so in these situations we
require:
 * an extremely clear commit message about why this is required (and why
backwards compatible behavior is not an option)
 * a link in the commit message to the dependent review (you can just
put the idempotent change id in there)
 * a +2 on the dependent review in the project by a core member

The 3rd part is important, because incompatible behavior changes are
something we want to make the core team for any given project is sure
they want to do before we drop or skip a test and let those changes come
into the project.

This system isn't perfect, but it's trying to strike a balance. My hope
is in the future we could implement cross project dependencies in zuul
so that you could gather test results for a project change - assuming
the tempest change was applied, that would prevent all these changes
from being -1ed until the tempest change is landed.

	-Sean

-- 
Sean Dague
Samsung Research America
sean at dague.net / sean.dague at samsung.com
http://dague.net

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 482 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140325/bbc7a839/attachment.pgp>