Open Stack

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

So cool! A comment inline.

On 02/10/2015 11:26 PM, James E. Blair wrote:
> Hi,
> 
> We have added support for cross-repo dependencies (CRD) in Zuul.
> The important bits:
> 
> * To use them, include "Depends-On: <gerrit-change-id>" in the
> footer of your commit message.  Use the full Change-ID ('I' + 40
> characters).
> 
> * These are one-way dependencies only -- do not create a cycle.
> 
> * This is what all the grey dots and lines are in the check
> pipeline.
> 
> Cross-Repo Dependencies Explained 
> =================================
> 
> There are two behaviors that might go by the name "cross-repo 
> dependencies".  We call them one-way and multi-way.
> 
> Multi-way CRD allow for bidirectional links between changes.  For 
> instance, A depends on B and B depends on A.  The theory there is
> that both would be tested together and merged as a unit.  This is
> _not_ what we have implemented in Zuul.  Discussions over the past
> two years have revealed that this type of behavior could cause
> problems for continuous deploments as it means that two components
> must be upgraded simultaneously.  Not supporting this behavior is a
> choice we have made.

Though I understand your reasoning behind not providing atomic merges
of multiple commits, I want to say that the feature would be very
useful for projects that are not decoupled enough. F.e. see neutron
advanced services repos that are split out of main neutron project but
still use lots of code from neutron namespace, making it hard to
introduce changes into the main repo without breaking advanced
services for some time. The same is true for decomposed plugins and
mechanism drivers for neutron.

> 
> One-way CRD behaves like a directed acyclic graph (DAG), like git 
> itself, to indicate a one-way dependency relationship between
> changes in different git repos.  Change A may depend on B, but B
> may not depend on A.  This is what we have implemented in Zuul.
> 
> Gate Pipeline =============
> 
> When Zuul sees CRD changes, it serializes them in the usual manner
> when enqueuing them into a pipeline.  This means that if change A
> depends on B, then when they are added to the gate pipeline, B will
> appear first and A will follow.  If tests for B fail, both B and A
> will be removed from the pipeline, and it will not be possible for
> A to merge until B does.
> 
> Note that if changes with CRD do not share a change queue (such as
> the "integrated gate" then Zuul is unable to enqueue them together,
> and the first will be required to merge before the second is
> enqueued.
> 
> Check Pipeline ==============
> 
> When changes are enqueued into the check pipeline, all of the
> related dependencies (both normal git-dependencies that come from
> parent commits as well as CRD changes) appear in a dependency
> graph, as in gate.  This means that even in the check pipeline,
> your change will be tested with its dependency.  So changes that
> were previously unable to be fully tested until a related change
> landed in a different repo may now be tested together from the
> start.
> 
> All of the changes are still independent (so you will note that
> the whole pipeline does not share a graph as in gate), but for each
> change tested, all of its dependencies are visually connected to
> it, and they are used to construct the git references that Zuul
> uses when testing. When looking at this graph on the status page,
> you will note that the dependencies show up as grey dots, while the
> actual change tested shows up as red or green.  This is to indicate
> that the grey changes are only there to establish dependencies.
> Even if one of the dependencies is also being tested, it will show
> up as a grey dot when used as a dependency, but separately and
> additionally will appear as its own red or green dot for its test.
> 
> (If you don't see grey dots on the status page, reload the page to
> get the latest version.)
> 
> Multiple Changes ================
> 
> A Gerrit change ID may refer to multiple changes (on multiple
> branches of the same project, or even multiple projects).  In these
> cases, Zuul will treat all of the changes with that change ID as
> dependencies.  So if you say that a tempest change Depends-On a
> change ID that has changes in nova master and nova stable/juno,
> then when testing the tempest change, both nova changes will be
> applied, and when deciding whether the tempest change can merge,
> both changes must merge ahead of it.
> 
> A change may depend on more than one Gerrit change ID as well.  So
> it is possible for a change in tempest to depend on a change in
> devstack and a change in nova.  Simply add more "Depends-On:" lines
> to the footer.
> 
> Cycles ======
> 
> If a cycle is created by use of CRD, Zuul will abort its work very 
> early.  There will be no message in Gerrit and no changes that are
> part of the cycle will be enqueued into any pipeline.  This is to
> protect Zuul from infinite loops.  I hope that we can improve this
> to at least leave a message in Gerrit in the future.  But in the
> meantime, please be cognizant of this and do not create dependency
> cycles with Depends-On lines.
> 
> Examples ========
> 
> The following two infra changes have been tested together because
> of the Depends-On: line in the commit message of the first:
> 
> https://review.openstack.org/#/c/152508/ 
> https://review.openstack.org/#/c/152504/
> 
> In fact, you can see earlier test results failing until it was
> rechecked after CRD went into production (around 2015-02-10 15:20
> UTC).
> 
> This devstack change depended on a grenade change:
> 
> https://review.openstack.org/#/c/154575/ 
> https://review.openstack.org/#/c/153702/
> 
> And with CRD, was able to be tested in check together as well as
> being enqueued into the gate at the same time as its dependency.
> 
> Note that in this example, the Gerrit change ID that 154575 depends
> on is actually associated with two changes on two separate
> branches.  In cases such as this, Zuul treats all instances as
> dependencies (so both changes would be tested together, and both
> must merge before 154575 may).
> 
> Further Questions =================
> 
> This is a fairly substantial new feature, and we may still have a
> bit to learn about it.
> 
> If you have any further questions, feel free to ask here or in the 
> #openstack-infra channel on Freenode.
> 
> We will update the infra manual soon to reflect this change.
> 
> -Jim
> 
> __________________________________________________________________________
>
> 
OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe 
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBAgAGBQJU2op/AAoJEC5aWaUY1u571L0IANGtfsahmP6qx9mOut8gbok+
5OFYw7b8fUc3H9dX8L2Gp7BlDSR4F//Q9wH0ortNtmFIQfMSb9uzAbA2E8ro8D7i
3nYhpeMRsrkzSIuvLUXJJxlU4T+zLABvG6brfL3N/+lmgnk6mmvGztCZtRabne/Y
ThjfzXO7lD1HmLZHXmpRe3HhWAmEy91VsV0nqn407vdX7MJh7Zo+BnQlbtxN3804
FX+cd6nmTKPGRQaRBnXnTrAy2L877dFY7Hf8ibvpZ5gFH6ukjFmuCzbGpdGx/EKa
pRCidLmo1azp6jLZGyNKm/dWAMQ5JeH2tQlEFQ0GMeB5bkjc670tqwaTqLhL/1E=
=SdEx
-----END PGP SIGNATURE-----