[OpenStack-Infra] Cross-Repo Dependencies in Zuul

Anita Kuno anteaya at anteaya.info
Tue Feb 10 23:02:21 UTC 2015


On 02/10/2015 05:56 PM, Boris Pavlovic wrote:
> Anita,
> 
> Sorry for my bad english and not proper usage of "guys".
> But the only thing that I was trying to say is that infra team rocks.
> (that's all)
> 
> 
> Best regards,
> Boris Pavlovic
Great, thanks Boris.

Anita.
> 
> On Wed, Feb 11, 2015 at 1:44 AM, Anita Kuno <anteaya at anteaya.info> wrote:
> 
>> On 02/10/2015 05:42 PM, Boris Pavlovic wrote:
>>> Anita,
>>>
>>>
>>> By "guys" I refer to whole infra team including women...
>>> P.S. I thought that "guys" is gender-neutral....
>> It isn't.
>>
>> Folks, group, team are all gender neutral.
>>
>> A recent tc meeting likes 'y'all' for group gender neutral terms.
>>
>> Thanks,
>> Anita.
>>>
>>>
>>> Best regards,
>>> Boris Pavlovic
>>>
>>> On Wed, Feb 11, 2015 at 1:34 AM, Anita Kuno <anteaya at anteaya.info>
>> wrote:
>>>
>>>> On 02/10/2015 05:26 PM, Boris Pavlovic wrote:
>>>>> James,
>>>>>
>>>>> Awesome! Amazing! You guys rock!=)
>>>> Hopefully the women do too.
>>>>
>>>> Anita.
>>>>>
>>>>> Best regards,
>>>>> Boris Pavlovic
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Feb 11, 2015 at 1:26 AM, James E. Blair <corvus at inaugust.com>
>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> We have added support for cross-repo dependencies (CRD) in Zuul.  The
>>>>>> important bits:
>>>>>>
>>>>>> * To use them, include "Depends-On: <gerrit-change-id>" in the footer
>> of
>>>>>>   your commit message.  Use the full Change-ID ('I' + 40 characters).
>>>>>>
>>>>>> * These are one-way dependencies only -- do not create a cycle.
>>>>>>
>>>>>> * This is what all the grey dots and lines are in the check pipeline.
>>>>>>
>>>>>> Cross-Repo Dependencies Explained
>>>>>> =================================
>>>>>>
>>>>>> There are two behaviors that might go by the name "cross-repo
>>>>>> dependencies".  We call them one-way and multi-way.
>>>>>>
>>>>>> Multi-way CRD allow for bidirectional links between changes.  For
>>>>>> instance, A depends on B and B depends on A.  The theory there is that
>>>>>> both would be tested together and merged as a unit.  This is _not_
>> what
>>>>>> we have implemented in Zuul.  Discussions over the past two years have
>>>>>> revealed that this type of behavior could cause problems for
>> continuous
>>>>>> deploments as it means that two components must be upgraded
>>>>>> simultaneously.  Not supporting this behavior is a choice we have
>> made.
>>>>>>
>>>>>> One-way CRD behaves like a directed acyclic graph (DAG), like git
>>>>>> itself, to indicate a one-way dependency relationship between changes
>> in
>>>>>> different git repos.  Change A may depend on B, but B may not depend
>> on
>>>>>> A.  This is what we have implemented in Zuul.
>>>>>>
>>>>>> Gate Pipeline
>>>>>> =============
>>>>>>
>>>>>> When Zuul sees CRD changes, it serializes them in the usual manner
>> when
>>>>>> enqueuing them into a pipeline.  This means that if change A depends
>> on
>>>>>> B, then when they are added to the gate pipeline, B will appear first
>>>>>> and A will follow.  If tests for B fail, both B and A will be removed
>>>>>> from the pipeline, and it will not be possible for A to merge until B
>>>>>> does.
>>>>>>
>>>>>> Note that if changes with CRD do not share a change queue (such as the
>>>>>> "integrated gate" then Zuul is unable to enqueue them together, and
>> the
>>>>>> first will be required to merge before the second is enqueued.
>>>>>>
>>>>>> Check Pipeline
>>>>>> ==============
>>>>>>
>>>>>> When changes are enqueued into the check pipeline, all of the related
>>>>>> dependencies (both normal git-dependencies that come from parent
>> commits
>>>>>> as well as CRD changes) appear in a dependency graph, as in gate.
>> This
>>>>>> means that even in the check pipeline, your change will be tested with
>>>>>> its dependency.  So changes that were previously unable to be fully
>>>>>> tested until a related change landed in a different repo may now be
>>>>>> tested together from the start.
>>>>>>
>>>>>> All of the changes are still independent (so you will note that the
>>>>>> whole pipeline does not share a graph as in gate), but for each change
>>>>>> tested, all of its dependencies are visually connected to it, and they
>>>>>> are used to construct the git references that Zuul uses when testing.
>>>>>> When looking at this graph on the status page, you will note that the
>>>>>> dependencies show up as grey dots, while the actual change tested
>> shows
>>>>>> up as red or green.  This is to indicate that the grey changes are
>> only
>>>>>> there to establish dependencies.  Even if one of the dependencies is
>>>>>> also being tested, it will show up as a grey dot when used as a
>>>>>> dependency, but separately and additionally will appear as its own red
>>>>>> or green dot for its test.
>>>>>>
>>>>>> (If you don't see grey dots on the status page, reload the page to get
>>>>>> the latest version.)
>>>>>>
>>>>>> Multiple Changes
>>>>>> ================
>>>>>>
>>>>>> A Gerrit change ID may refer to multiple changes (on multiple branches
>>>>>> of the same project, or even multiple projects).  In these cases, Zuul
>>>>>> will treat all of the changes with that change ID as dependencies.  So
>>>>>> if you say that a tempest change Depends-On a change ID that has
>> changes
>>>>>> in nova master and nova stable/juno, then when testing the tempest
>>>>>> change, both nova changes will be applied, and when deciding whether
>> the
>>>>>> tempest change can merge, both changes must merge ahead of it.
>>>>>>
>>>>>> A change may depend on more than one Gerrit change ID as well.  So it
>> is
>>>>>> possible for a change in tempest to depend on a change in devstack
>> and a
>>>>>> change in nova.  Simply add more "Depends-On:" lines to the footer.
>>>>>>
>>>>>> Cycles
>>>>>> ======
>>>>>>
>>>>>> If a cycle is created by use of CRD, Zuul will abort its work very
>>>>>> early.  There will be no message in Gerrit and no changes that are
>> part
>>>>>> of the cycle will be enqueued into any pipeline.  This is to protect
>>>>>> Zuul from infinite loops.  I hope that we can improve this to at least
>>>>>> leave a message in Gerrit in the future.  But in the meantime, please
>> be
>>>>>> cognizant of this and do not create dependency cycles with Depends-On
>>>>>> lines.
>>>>>>
>>>>>> Examples
>>>>>> ========
>>>>>>
>>>>>> The following two infra changes have been tested together because of
>> the
>>>>>> Depends-On: line in the commit message of the first:
>>>>>>
>>>>>>   https://review.openstack.org/#/c/152508/
>>>>>>   https://review.openstack.org/#/c/152504/
>>>>>>
>>>>>> In fact, you can see earlier test results failing until it was
>> rechecked
>>>>>> after CRD went into production (around 2015-02-10 15:20 UTC).
>>>>>>
>>>>>> This devstack change depended on a grenade change:
>>>>>>
>>>>>>   https://review.openstack.org/#/c/154575/
>>>>>>   https://review.openstack.org/#/c/153702/
>>>>>>
>>>>>> And with CRD, was able to be tested in check together as well as being
>>>>>> enqueued into the gate at the same time as its dependency.
>>>>>>
>>>>>> Note that in this example, the Gerrit change ID that 154575 depends on
>>>>>> is actually associated with two changes on two separate branches.  In
>>>>>> cases such as this, Zuul treats all instances as dependencies (so both
>>>>>> changes would be tested together, and both must merge before 154575
>>>>>> may).
>>>>>>
>>>>>> Further Questions
>>>>>> =================
>>>>>>
>>>>>> This is a fairly substantial new feature, and we may still have a bit
>> to
>>>>>> learn about it.
>>>>>>
>>>>>> If you have any further questions, feel free to ask here or in the
>>>>>> #openstack-infra channel on Freenode.
>>>>>>
>>>>>> We will update the infra manual soon to reflect this change.
>>>>>>
>>>>>> -Jim
>>>>>>
>>>>>> _______________________________________________
>>>>>> OpenStack-Infra mailing list
>>>>>> OpenStack-Infra at lists.openstack.org
>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> OpenStack-Infra mailing list
>>>>> OpenStack-Infra at lists.openstack.org
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> OpenStack-Infra mailing list
>>>> OpenStack-Infra at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>>>>
>>>
>>
>>
> 




More information about the OpenStack-Infra mailing list