Open Stack

On 28.02.2019 12:30, Bogdan Dobrelya wrote:
> Here is example jobs [0],[1] to illustrate the "middle-ground proposal".
> 
> As you can see, disregard of the failed tox, a few jobs will be executed:
> - tripleo-ci-centos-7-undercloud-containers (the base job for UC 
> deployments checking)
> - tripleo-ci-centos-7-standalone (the base job that emulates overcloud 
> deployments but on all-in-one standalone layout)
> - tripleo-ci-fedora-28-standalone (the same, but performed for f28 zuul 
> subnodes)

what's interesting, it gives even more prominent feedback for the listed 
"base level" checks, than it normally does. See, a 1h after the last 
update of [1], I have already been gotten some results this early. And 
for the 2nd DNM test [0], that took even shorter - only a 30 min.

> 
> So that hopefully still gives a developer some insights and as well 
> prevents the rest of the deemed to fail (or deprecated multinode) jobs 
> from execution and still saves some CI pool resources.
> 
> To illustrate the further ordering, see [2],[3] that is expected to have 
> the standalone and UC jobs passing. That would in turn cause the 
> update/upgrade and custom standalone scenarios jobs executed *after* 
> that logical build-step, so let's wait and see for results. If you think 
> we should limit the changes scope for dependencies of tox jobs only, let 
> me know and I'll remove those additional inter-jobs dependencies odd the 
> patches.

Note that [2],[3] takes longer comparing to [0],[1] as the former jobs 
are running after the "next level". So we can expect the total time of 
the full check pipeline will take longer by a 50 minutes or so.

And if we moved the update/upgrade jobs to the base level, we'd have 
results for those jobs always listed disregard of the tox jobs. But for 
that case, the base later would take a ~2h instead, therefore the total 
check pipeline delay would be also extended by that value.

> 
> PS. I don't think that reworked layout abuses zuul dependencies feature 
> in any way as we do have some logical state shared here across these 
> consequently executed jobs. That is only the "succeeded-or-not" flag so 
> far :-) Ideally, we'll need some real deployment artifacts shared, like 
> the updated containers registry.
> 
> [0] https://review.openstack.org/639615
> [1] https://review.openstack.org/639721
> [2] https://review.openstack.org/639725
> [3] https://review.openstack.org/639604
> 
> 
> On 27.02.2019 18:31, Bogdan Dobrelya wrote:
>> I think we can still consider the middle-ground, where only deprecated 
>> multinode jobs, which tripleo infra team is in progress of migrating 
>> into standalone jobs, could be made depending on unit and pep8 checks? 
>> And some basic jobs will keep being depending on nothing.
>>
>> I expanded that idea in WIP topic [0]. Commit messages explain how the 
>> ordering was reworked.
>>
>> PS. I'm sorry I missed the submitted stats for zuul projects posted 
>> earlier in this topic, I'll take a look into that.
>>
>> [0] 
>> https://review.openstack.org/#/q/topic:ci_pipelines+(status:open+OR+status:merged) 
>>
>>
>>> Bogdan Dobrelya <bdobreli at redhat.com> writes:
>>>> On 26.02.2019 17:53, James E. Blair wrote:
>>>>> Bogdan Dobrelya <bdobreli at redhat.com> writes:
>>>>>
>>>>>> I attempted [0] to do that for tripleo-ci, but zuul was (and still
>>>>>> does) complaining for some weird graphs building things :/
>>>>>>
>>>>>> See also the related topic [1] from the past.
>>>>>>
>>>>>> [0] https://review.openstack.org/#/c/568543
>>>>>> [1] 
>>>>>> http://lists.openstack.org/pipermail/openstack-dev/2018-March/127869.html 
>>>>>>
>>>>>
>>>>> Thank you for linking to [1].  It's worth re-reading.  Especially the
>>>>> part at the end.
>>>>>
>>>>> -Jim
>>>>>
>>>>
>>>
>>> Yes, the part at the end is the best indeed.
>>> I'd amend the time priorities graph though like that:
>>>
>>> CPU-time < a developer time < developers time
>>>
>>> That means burning some CPU and nodes in a pool for a waste might 
>>> benefit a developer, but saving some CPU and nodes in a pool would 
>>> benefit *developers* in many projects as they'd get the jobs results 
>>> off the waiting check queues faster :)
>>
>>
>>
> 
> 


-- 
Best regards,
Bogdan Dobrelya,
Irc #bogdando