Open Stack

Hello together,

Thanks for your answers.

Yes, we are interested in this feature and ready to contribute. But we 
will need a support from your side for clear understanding how to 
implement it.

My vision is a following:

Define a new configuration parameter for 'manager: 
DependentPipelineManager', e.g. 'manager-policy: aggregate-gerrit-topics'.

When change set arrives DependentPipelineManager and topic is not empty, 
consult gerrit server whether other change sets effecting pipeline 
exists. And if it is a case, delay a pipeline until all change sets with 
the same topic are ready to be proceeded.

WBR,
Oleksandr

"Those who cannot remember the past are condemned to repeat it"

George Santayana

"And those who do remember the past are condemned to stand by helplessly while everyone else repeats it."

Am 23.01.2017 um 19:15 schrieb James E. Blair:
> "Karpenko, Oleksandr (Nokia - DE/Ulm)" <oleksandr.karpenko at nokia.com>
> writes:
>
>> Hallo together,
>>
>> Sorry for bothering you but I am not able to find any zuul specific mailing list.
>> First of all if there is some better place to ask questions, please
>> let me know and I will do it there.
> You've come to the right place.
>
>> We are considering to use Zuul in a project split across many git
>> repositories and we are in a proof of concept phase now. I have
>> submitted for the review first problems we have found
>> (https://review.openstack.org/#/c/423337/ ,
>> https://review.openstack.org/#/c/424055/).
> Thanks for those changes to the gearman-plugin for Jenkins.  I think the
> first especially may be of use to a lot of people who have run into
> permissions problems with newer Jenkins.
>
>> Gerrit has a nice feature called 'Submit whole topic'. You can have
>> reviews in several repositories within the same topic and gerrit
>> allows to submit them together very close in time. When the first
>> change is ready for Submit, 'Submit' button stays gray until all other
>> reviews in the same topic are ready and then by pressing the button in
>> any of those changes all of them are submitted.
>>
>> We have gate pipeline (DependentPipeline) in Zuul for Cross Project
>> Testing which is triggered on code-review: 2 and on success does
>> gerrit submit. The gate pipeline runs when the first change gets
>> code-review: 2 and if all other changes have also code-review: 2, zuul
>> submits all of them without testing the rest.
>>
>> Does Zuul have any support for 'Submit whole topic' gerrit feature? I
>> am not able to find any.
>>
>> Does DependentPipeline have support for gerrit topic concept? It
>> sounds at least reasonable to wait until all changes in the same topic
>> get reviewed and then start DependentPipeline once (instead of 3 times
>> in case of 2 repositories).
>>
>> Does our concept make sense at all or there are other ways to do it?
>> We would like to reduce an effect of 'diamond dependency problem' by
>> submitting related to each other changes close in time. For example
>> when A depends on B and C. B and C depends on D. We would like to test
>> A with related to each other changes in B and C (e.g. D integration)
>> and submit changes in B and C very close in time. Because when B
>> integrates D is much faster as C, A cannot be built until C is ready
>> with D integration. This is not exactly what 'Depends-On:' is doing
>> because changes in B and C does not depend on each other.
>>
>> Once more sorry for your time and looking forward for your answer.
> Clint replied earlier with some good and correct information -- here is
> a little more context and some thoughts about future development.
>
> We implemented the Depens-On header feature in Zuul based on some
> earlier work on cross-repo dependencies in Gerrit.  We have found the
> 'topic' to be useful in gerrit for far more things than just indicating
> changes that should be merged together, and I personally am sad to see
> it used for cross-repo dependencies as it robs Gerrit users of a useful
> tool.
>
> Regardless, you are correct that we only support one-way dependencies in
> Zuul.  That is because we decided that, for the OpenStack project, we
> wanted to avoid changing two repositories at once.  Many of our projects
> are continuously deployed, and an OpenStack installation spans many
> hosts (and also contains client-server relationships).  Therefore, we
> wanted to ensure that developers changed each component in such a way
> that it was backwards-compatible with other components.  This is
> especially useful in the environment that I described, but also ends up
> being good practice in more traditional application + library
> situations.
>
> However, I can see that some environments might prefer bidirectional
> cross-repo dependencies.  Many of the building blocks needed to develop
> that are already in zuul (we wrote the cross-repo-dependency code with
> the idea that we might extend it to support this case in the future),
> but there is still some work remaining to implement that.
>
> If one were to implement it, I would recommend continuing to avoid using
> the submit-whole-topic feature in Gerrit (as it is specific to Gerrit
> and therefore not applicable to other systems with which we want to
> interface Zuul in the future, and also, it is not necessary for this
> work).  An implementation in Zuul should simply test both repos with the
> state of the other, and then push the prepared repo states that it
> tested up to Gerrit rather than 'submitting' the change using Gerrit.
> This would allow Zuul to be certain that both merges will succeed (since
> it has already performed them locally).
>
> I think that we will end up implementing this eventually, but it is not
> high on our list of priorities at the moment.  If you are interested in
> working on this sooner, please let me know.
>
> Finally, I would like to note that since you are in the exploration
> stage that we are currently working heavily on version 3 of Zuul which
> uses Ansible for running jobs rather than Jenkins.  We expect it to be
> just as flexible terms of job content (if not more so), and while it's
> not certain how much we will be able to automate this, we do expect to
> have some tools for migrating some jenkins-job-builder configurations to
> Ansible.  It's worth being aware of the direction we are heading as you
> look into using Zuul.
>
> Thanks,
>
> Jim