[OpenStack-Infra] Cross-Repo Dependencies in Zuul

Boris Pavlovic boris at pavlovic.me
Tue Feb 10 22:56:31 UTC 2015


Anita,

Sorry for my bad english and not proper usage of "guys".
But the only thing that I was trying to say is that infra team rocks.
(that's all)


Best regards,
Boris Pavlovic

On Wed, Feb 11, 2015 at 1:44 AM, Anita Kuno <anteaya at anteaya.info> wrote:

> On 02/10/2015 05:42 PM, Boris Pavlovic wrote:
> > Anita,
> >
> >
> > By "guys" I refer to whole infra team including women...
> > P.S. I thought that "guys" is gender-neutral....
> It isn't.
>
> Folks, group, team are all gender neutral.
>
> A recent tc meeting likes 'y'all' for group gender neutral terms.
>
> Thanks,
> Anita.
> >
> >
> > Best regards,
> > Boris Pavlovic
> >
> > On Wed, Feb 11, 2015 at 1:34 AM, Anita Kuno <anteaya at anteaya.info>
> wrote:
> >
> >> On 02/10/2015 05:26 PM, Boris Pavlovic wrote:
> >>> James,
> >>>
> >>> Awesome! Amazing! You guys rock!=)
> >> Hopefully the women do too.
> >>
> >> Anita.
> >>>
> >>> Best regards,
> >>> Boris Pavlovic
> >>>
> >>>
> >>>
> >>> On Wed, Feb 11, 2015 at 1:26 AM, James E. Blair <corvus at inaugust.com>
> >> wrote:
> >>>
> >>>> Hi,
> >>>>
> >>>> We have added support for cross-repo dependencies (CRD) in Zuul.  The
> >>>> important bits:
> >>>>
> >>>> * To use them, include "Depends-On: <gerrit-change-id>" in the footer
> of
> >>>>   your commit message.  Use the full Change-ID ('I' + 40 characters).
> >>>>
> >>>> * These are one-way dependencies only -- do not create a cycle.
> >>>>
> >>>> * This is what all the grey dots and lines are in the check pipeline.
> >>>>
> >>>> Cross-Repo Dependencies Explained
> >>>> =================================
> >>>>
> >>>> There are two behaviors that might go by the name "cross-repo
> >>>> dependencies".  We call them one-way and multi-way.
> >>>>
> >>>> Multi-way CRD allow for bidirectional links between changes.  For
> >>>> instance, A depends on B and B depends on A.  The theory there is that
> >>>> both would be tested together and merged as a unit.  This is _not_
> what
> >>>> we have implemented in Zuul.  Discussions over the past two years have
> >>>> revealed that this type of behavior could cause problems for
> continuous
> >>>> deploments as it means that two components must be upgraded
> >>>> simultaneously.  Not supporting this behavior is a choice we have
> made.
> >>>>
> >>>> One-way CRD behaves like a directed acyclic graph (DAG), like git
> >>>> itself, to indicate a one-way dependency relationship between changes
> in
> >>>> different git repos.  Change A may depend on B, but B may not depend
> on
> >>>> A.  This is what we have implemented in Zuul.
> >>>>
> >>>> Gate Pipeline
> >>>> =============
> >>>>
> >>>> When Zuul sees CRD changes, it serializes them in the usual manner
> when
> >>>> enqueuing them into a pipeline.  This means that if change A depends
> on
> >>>> B, then when they are added to the gate pipeline, B will appear first
> >>>> and A will follow.  If tests for B fail, both B and A will be removed
> >>>> from the pipeline, and it will not be possible for A to merge until B
> >>>> does.
> >>>>
> >>>> Note that if changes with CRD do not share a change queue (such as the
> >>>> "integrated gate" then Zuul is unable to enqueue them together, and
> the
> >>>> first will be required to merge before the second is enqueued.
> >>>>
> >>>> Check Pipeline
> >>>> ==============
> >>>>
> >>>> When changes are enqueued into the check pipeline, all of the related
> >>>> dependencies (both normal git-dependencies that come from parent
> commits
> >>>> as well as CRD changes) appear in a dependency graph, as in gate.
> This
> >>>> means that even in the check pipeline, your change will be tested with
> >>>> its dependency.  So changes that were previously unable to be fully
> >>>> tested until a related change landed in a different repo may now be
> >>>> tested together from the start.
> >>>>
> >>>> All of the changes are still independent (so you will note that the
> >>>> whole pipeline does not share a graph as in gate), but for each change
> >>>> tested, all of its dependencies are visually connected to it, and they
> >>>> are used to construct the git references that Zuul uses when testing.
> >>>> When looking at this graph on the status page, you will note that the
> >>>> dependencies show up as grey dots, while the actual change tested
> shows
> >>>> up as red or green.  This is to indicate that the grey changes are
> only
> >>>> there to establish dependencies.  Even if one of the dependencies is
> >>>> also being tested, it will show up as a grey dot when used as a
> >>>> dependency, but separately and additionally will appear as its own red
> >>>> or green dot for its test.
> >>>>
> >>>> (If you don't see grey dots on the status page, reload the page to get
> >>>> the latest version.)
> >>>>
> >>>> Multiple Changes
> >>>> ================
> >>>>
> >>>> A Gerrit change ID may refer to multiple changes (on multiple branches
> >>>> of the same project, or even multiple projects).  In these cases, Zuul
> >>>> will treat all of the changes with that change ID as dependencies.  So
> >>>> if you say that a tempest change Depends-On a change ID that has
> changes
> >>>> in nova master and nova stable/juno, then when testing the tempest
> >>>> change, both nova changes will be applied, and when deciding whether
> the
> >>>> tempest change can merge, both changes must merge ahead of it.
> >>>>
> >>>> A change may depend on more than one Gerrit change ID as well.  So it
> is
> >>>> possible for a change in tempest to depend on a change in devstack
> and a
> >>>> change in nova.  Simply add more "Depends-On:" lines to the footer.
> >>>>
> >>>> Cycles
> >>>> ======
> >>>>
> >>>> If a cycle is created by use of CRD, Zuul will abort its work very
> >>>> early.  There will be no message in Gerrit and no changes that are
> part
> >>>> of the cycle will be enqueued into any pipeline.  This is to protect
> >>>> Zuul from infinite loops.  I hope that we can improve this to at least
> >>>> leave a message in Gerrit in the future.  But in the meantime, please
> be
> >>>> cognizant of this and do not create dependency cycles with Depends-On
> >>>> lines.
> >>>>
> >>>> Examples
> >>>> ========
> >>>>
> >>>> The following two infra changes have been tested together because of
> the
> >>>> Depends-On: line in the commit message of the first:
> >>>>
> >>>>   https://review.openstack.org/#/c/152508/
> >>>>   https://review.openstack.org/#/c/152504/
> >>>>
> >>>> In fact, you can see earlier test results failing until it was
> rechecked
> >>>> after CRD went into production (around 2015-02-10 15:20 UTC).
> >>>>
> >>>> This devstack change depended on a grenade change:
> >>>>
> >>>>   https://review.openstack.org/#/c/154575/
> >>>>   https://review.openstack.org/#/c/153702/
> >>>>
> >>>> And with CRD, was able to be tested in check together as well as being
> >>>> enqueued into the gate at the same time as its dependency.
> >>>>
> >>>> Note that in this example, the Gerrit change ID that 154575 depends on
> >>>> is actually associated with two changes on two separate branches.  In
> >>>> cases such as this, Zuul treats all instances as dependencies (so both
> >>>> changes would be tested together, and both must merge before 154575
> >>>> may).
> >>>>
> >>>> Further Questions
> >>>> =================
> >>>>
> >>>> This is a fairly substantial new feature, and we may still have a bit
> to
> >>>> learn about it.
> >>>>
> >>>> If you have any further questions, feel free to ask here or in the
> >>>> #openstack-infra channel on Freenode.
> >>>>
> >>>> We will update the infra manual soon to reflect this change.
> >>>>
> >>>> -Jim
> >>>>
> >>>> _______________________________________________
> >>>> OpenStack-Infra mailing list
> >>>> OpenStack-Infra at lists.openstack.org
> >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
> >>>>
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> OpenStack-Infra mailing list
> >>> OpenStack-Infra at lists.openstack.org
> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
> >>>
> >>
> >>
> >> _______________________________________________
> >> OpenStack-Infra mailing list
> >> OpenStack-Infra at lists.openstack.org
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
> >>
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-infra/attachments/20150211/c13c291c/attachment-0001.html>


More information about the OpenStack-Infra mailing list