[OpenStack-Infra] Cross-Repo Dependencies in Zuul

Anita Kuno anteaya at anteaya.info
Tue Feb 10 22:44:24 UTC 2015


On 02/10/2015 05:42 PM, Boris Pavlovic wrote:
> Anita,
> 
> 
> By "guys" I refer to whole infra team including women...
> P.S. I thought that "guys" is gender-neutral....
It isn't.

Folks, group, team are all gender neutral.

A recent tc meeting likes 'y'all' for group gender neutral terms.

Thanks,
Anita.
> 
> 
> Best regards,
> Boris Pavlovic
> 
> On Wed, Feb 11, 2015 at 1:34 AM, Anita Kuno <anteaya at anteaya.info> wrote:
> 
>> On 02/10/2015 05:26 PM, Boris Pavlovic wrote:
>>> James,
>>>
>>> Awesome! Amazing! You guys rock!=)
>> Hopefully the women do too.
>>
>> Anita.
>>>
>>> Best regards,
>>> Boris Pavlovic
>>>
>>>
>>>
>>> On Wed, Feb 11, 2015 at 1:26 AM, James E. Blair <corvus at inaugust.com>
>> wrote:
>>>
>>>> Hi,
>>>>
>>>> We have added support for cross-repo dependencies (CRD) in Zuul.  The
>>>> important bits:
>>>>
>>>> * To use them, include "Depends-On: <gerrit-change-id>" in the footer of
>>>>   your commit message.  Use the full Change-ID ('I' + 40 characters).
>>>>
>>>> * These are one-way dependencies only -- do not create a cycle.
>>>>
>>>> * This is what all the grey dots and lines are in the check pipeline.
>>>>
>>>> Cross-Repo Dependencies Explained
>>>> =================================
>>>>
>>>> There are two behaviors that might go by the name "cross-repo
>>>> dependencies".  We call them one-way and multi-way.
>>>>
>>>> Multi-way CRD allow for bidirectional links between changes.  For
>>>> instance, A depends on B and B depends on A.  The theory there is that
>>>> both would be tested together and merged as a unit.  This is _not_ what
>>>> we have implemented in Zuul.  Discussions over the past two years have
>>>> revealed that this type of behavior could cause problems for continuous
>>>> deploments as it means that two components must be upgraded
>>>> simultaneously.  Not supporting this behavior is a choice we have made.
>>>>
>>>> One-way CRD behaves like a directed acyclic graph (DAG), like git
>>>> itself, to indicate a one-way dependency relationship between changes in
>>>> different git repos.  Change A may depend on B, but B may not depend on
>>>> A.  This is what we have implemented in Zuul.
>>>>
>>>> Gate Pipeline
>>>> =============
>>>>
>>>> When Zuul sees CRD changes, it serializes them in the usual manner when
>>>> enqueuing them into a pipeline.  This means that if change A depends on
>>>> B, then when they are added to the gate pipeline, B will appear first
>>>> and A will follow.  If tests for B fail, both B and A will be removed
>>>> from the pipeline, and it will not be possible for A to merge until B
>>>> does.
>>>>
>>>> Note that if changes with CRD do not share a change queue (such as the
>>>> "integrated gate" then Zuul is unable to enqueue them together, and the
>>>> first will be required to merge before the second is enqueued.
>>>>
>>>> Check Pipeline
>>>> ==============
>>>>
>>>> When changes are enqueued into the check pipeline, all of the related
>>>> dependencies (both normal git-dependencies that come from parent commits
>>>> as well as CRD changes) appear in a dependency graph, as in gate.  This
>>>> means that even in the check pipeline, your change will be tested with
>>>> its dependency.  So changes that were previously unable to be fully
>>>> tested until a related change landed in a different repo may now be
>>>> tested together from the start.
>>>>
>>>> All of the changes are still independent (so you will note that the
>>>> whole pipeline does not share a graph as in gate), but for each change
>>>> tested, all of its dependencies are visually connected to it, and they
>>>> are used to construct the git references that Zuul uses when testing.
>>>> When looking at this graph on the status page, you will note that the
>>>> dependencies show up as grey dots, while the actual change tested shows
>>>> up as red or green.  This is to indicate that the grey changes are only
>>>> there to establish dependencies.  Even if one of the dependencies is
>>>> also being tested, it will show up as a grey dot when used as a
>>>> dependency, but separately and additionally will appear as its own red
>>>> or green dot for its test.
>>>>
>>>> (If you don't see grey dots on the status page, reload the page to get
>>>> the latest version.)
>>>>
>>>> Multiple Changes
>>>> ================
>>>>
>>>> A Gerrit change ID may refer to multiple changes (on multiple branches
>>>> of the same project, or even multiple projects).  In these cases, Zuul
>>>> will treat all of the changes with that change ID as dependencies.  So
>>>> if you say that a tempest change Depends-On a change ID that has changes
>>>> in nova master and nova stable/juno, then when testing the tempest
>>>> change, both nova changes will be applied, and when deciding whether the
>>>> tempest change can merge, both changes must merge ahead of it.
>>>>
>>>> A change may depend on more than one Gerrit change ID as well.  So it is
>>>> possible for a change in tempest to depend on a change in devstack and a
>>>> change in nova.  Simply add more "Depends-On:" lines to the footer.
>>>>
>>>> Cycles
>>>> ======
>>>>
>>>> If a cycle is created by use of CRD, Zuul will abort its work very
>>>> early.  There will be no message in Gerrit and no changes that are part
>>>> of the cycle will be enqueued into any pipeline.  This is to protect
>>>> Zuul from infinite loops.  I hope that we can improve this to at least
>>>> leave a message in Gerrit in the future.  But in the meantime, please be
>>>> cognizant of this and do not create dependency cycles with Depends-On
>>>> lines.
>>>>
>>>> Examples
>>>> ========
>>>>
>>>> The following two infra changes have been tested together because of the
>>>> Depends-On: line in the commit message of the first:
>>>>
>>>>   https://review.openstack.org/#/c/152508/
>>>>   https://review.openstack.org/#/c/152504/
>>>>
>>>> In fact, you can see earlier test results failing until it was rechecked
>>>> after CRD went into production (around 2015-02-10 15:20 UTC).
>>>>
>>>> This devstack change depended on a grenade change:
>>>>
>>>>   https://review.openstack.org/#/c/154575/
>>>>   https://review.openstack.org/#/c/153702/
>>>>
>>>> And with CRD, was able to be tested in check together as well as being
>>>> enqueued into the gate at the same time as its dependency.
>>>>
>>>> Note that in this example, the Gerrit change ID that 154575 depends on
>>>> is actually associated with two changes on two separate branches.  In
>>>> cases such as this, Zuul treats all instances as dependencies (so both
>>>> changes would be tested together, and both must merge before 154575
>>>> may).
>>>>
>>>> Further Questions
>>>> =================
>>>>
>>>> This is a fairly substantial new feature, and we may still have a bit to
>>>> learn about it.
>>>>
>>>> If you have any further questions, feel free to ask here or in the
>>>> #openstack-infra channel on Freenode.
>>>>
>>>> We will update the infra manual soon to reflect this change.
>>>>
>>>> -Jim
>>>>
>>>> _______________________________________________
>>>> OpenStack-Infra mailing list
>>>> OpenStack-Infra at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> OpenStack-Infra mailing list
>>> OpenStack-Infra at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>>>
>>
>>
>> _______________________________________________
>> OpenStack-Infra mailing list
>> OpenStack-Infra at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>>
> 




More information about the OpenStack-Infra mailing list