Re: [placement][TripleO][infra] zuul job dependencies for greater good?
I think we can still consider the middle-ground, where only deprecated multinode jobs, which tripleo infra team is in progress of migrating into standalone jobs, could be made depending on unit and pep8 checks? And some basic jobs will keep being depending on nothing.
I expanded that idea in WIP topic [0]. Commit messages explain how the ordering was reworked.
PS. I'm sorry I missed the submitted stats for zuul projects posted earlier in this topic, I'll take a look into that.
[0] https://review.openstack.org/#/q/topic:ci_pipelines+(status:open+OR+status:m...)
Bogdan Dobrelya <bdobreli at redhat.com> writes:
On 26.02.2019 17:53, James E. Blair wrote:
Bogdan Dobrelya <bdobreli at redhat.com> writes:
I attempted [0] to do that for tripleo-ci, but zuul was (and still does) complaining for some weird graphs building things :/
See also the related topic [1] from the past.
[0] https://review.openstack.org/#/c/568543 [1] http://lists.openstack.org/pipermail/openstack-dev/2018-March/127869.html
Thank you for linking to [1]. It's worth re-reading. Especially the part at the end.
-Jim
Yes, the part at the end is the best indeed. I'd amend the time priorities graph though like that:
CPU-time < a developer time < developers time
That means burning some CPU and nodes in a pool for a waste might benefit a developer, but saving some CPU and nodes in a pool would benefit *developers* in many projects as they'd get the jobs results off the waiting check queues faster :)
Here is example jobs [0],[1] to illustrate the "middle-ground proposal".
As you can see, disregard of the failed tox, a few jobs will be executed: - tripleo-ci-centos-7-undercloud-containers (the base job for UC deployments checking) - tripleo-ci-centos-7-standalone (the base job that emulates overcloud deployments but on all-in-one standalone layout) - tripleo-ci-fedora-28-standalone (the same, but performed for f28 zuul subnodes)
So that hopefully still gives a developer some insights and as well prevents the rest of the deemed to fail (or deprecated multinode) jobs from execution and still saves some CI pool resources.
To illustrate the further ordering, see [2],[3] that is expected to have the standalone and UC jobs passing. That would in turn cause the update/upgrade and custom standalone scenarios jobs executed *after* that logical build-step, so let's wait and see for results. If you think we should limit the changes scope for dependencies of tox jobs only, let me know and I'll remove those additional inter-jobs dependencies odd the patches.
PS. I don't think that reworked layout abuses zuul dependencies feature in any way as we do have some logical state shared here across these consequently executed jobs. That is only the "succeeded-or-not" flag so far :-) Ideally, we'll need some real deployment artifacts shared, like the updated containers registry.
[0] https://review.openstack.org/639615 [1] https://review.openstack.org/639721 [2] https://review.openstack.org/639725 [3] https://review.openstack.org/639604
On 27.02.2019 18:31, Bogdan Dobrelya wrote:
I think we can still consider the middle-ground, where only deprecated multinode jobs, which tripleo infra team is in progress of migrating into standalone jobs, could be made depending on unit and pep8 checks? And some basic jobs will keep being depending on nothing.
I expanded that idea in WIP topic [0]. Commit messages explain how the ordering was reworked.
PS. I'm sorry I missed the submitted stats for zuul projects posted earlier in this topic, I'll take a look into that.
[0] https://review.openstack.org/#/q/topic:ci_pipelines+(status:open+OR+status:m...)
Bogdan Dobrelya <bdobreli at redhat.com> writes:
On 26.02.2019 17:53, James E. Blair wrote:
Bogdan Dobrelya <bdobreli at redhat.com> writes:
I attempted [0] to do that for tripleo-ci, but zuul was (and still does) complaining for some weird graphs building things :/
See also the related topic [1] from the past.
[0] https://review.openstack.org/#/c/568543 [1] http://lists.openstack.org/pipermail/openstack-dev/2018-March/127869.html
Thank you for linking to [1]. It's worth re-reading. Especially the part at the end.
-Jim
Yes, the part at the end is the best indeed. I'd amend the time priorities graph though like that:
CPU-time < a developer time < developers time
That means burning some CPU and nodes in a pool for a waste might benefit a developer, but saving some CPU and nodes in a pool would benefit *developers* in many projects as they'd get the jobs results off the waiting check queues faster :)
On 28.02.2019 12:30, Bogdan Dobrelya wrote:
Here is example jobs [0],[1] to illustrate the "middle-ground proposal".
As you can see, disregard of the failed tox, a few jobs will be executed:
- tripleo-ci-centos-7-undercloud-containers (the base job for UC
deployments checking)
- tripleo-ci-centos-7-standalone (the base job that emulates overcloud
deployments but on all-in-one standalone layout)
- tripleo-ci-fedora-28-standalone (the same, but performed for f28 zuul
subnodes)
what's interesting, it gives even more prominent feedback for the listed "base level" checks, than it normally does. See, a 1h after the last update of [1], I have already been gotten some results this early. And for the 2nd DNM test [0], that took even shorter - only a 30 min.
So that hopefully still gives a developer some insights and as well prevents the rest of the deemed to fail (or deprecated multinode) jobs from execution and still saves some CI pool resources.
To illustrate the further ordering, see [2],[3] that is expected to have the standalone and UC jobs passing. That would in turn cause the update/upgrade and custom standalone scenarios jobs executed *after* that logical build-step, so let's wait and see for results. If you think we should limit the changes scope for dependencies of tox jobs only, let me know and I'll remove those additional inter-jobs dependencies odd the patches.
Note that [2],[3] takes longer comparing to [0],[1] as the former jobs are running after the "next level". So we can expect the total time of the full check pipeline will take longer by a 50 minutes or so.
And if we moved the update/upgrade jobs to the base level, we'd have results for those jobs always listed disregard of the tox jobs. But for that case, the base later would take a ~2h instead, therefore the total check pipeline delay would be also extended by that value.
PS. I don't think that reworked layout abuses zuul dependencies feature in any way as we do have some logical state shared here across these consequently executed jobs. That is only the "succeeded-or-not" flag so far :-) Ideally, we'll need some real deployment artifacts shared, like the updated containers registry.
[0] https://review.openstack.org/639615 [1] https://review.openstack.org/639721 [2] https://review.openstack.org/639725 [3] https://review.openstack.org/639604
On 27.02.2019 18:31, Bogdan Dobrelya wrote:
I think we can still consider the middle-ground, where only deprecated multinode jobs, which tripleo infra team is in progress of migrating into standalone jobs, could be made depending on unit and pep8 checks? And some basic jobs will keep being depending on nothing.
I expanded that idea in WIP topic [0]. Commit messages explain how the ordering was reworked.
PS. I'm sorry I missed the submitted stats for zuul projects posted earlier in this topic, I'll take a look into that.
[0] https://review.openstack.org/#/q/topic:ci_pipelines+(status:open+OR+status:m...)
Bogdan Dobrelya <bdobreli at redhat.com> writes:
On 26.02.2019 17:53, James E. Blair wrote:
Bogdan Dobrelya <bdobreli at redhat.com> writes:
I attempted [0] to do that for tripleo-ci, but zuul was (and still does) complaining for some weird graphs building things :/
See also the related topic [1] from the past.
[0] https://review.openstack.org/#/c/568543 [1] http://lists.openstack.org/pipermail/openstack-dev/2018-March/127869.html
Thank you for linking to [1]. It's worth re-reading. Especially the part at the end.
-Jim
Yes, the part at the end is the best indeed. I'd amend the time priorities graph though like that:
CPU-time < a developer time < developers time
That means burning some CPU and nodes in a pool for a waste might benefit a developer, but saving some CPU and nodes in a pool would benefit *developers* in many projects as they'd get the jobs results off the waiting check queues faster :)
participants (1)
-
Bogdan Dobrelya