On 2021-02-04 22:29:35 +0200 (+0200), Dmitriy Rabotyagov wrote:
For OSA huge issue is how zuul clones required-projects. Just this single action takes for us from 6 to 10 minutes. [...]
I'd be curious to see some examples of this. Zuul doesn't clone required-projects, but it does push new commits from a cache on the executor to a cache on the node. The executor side caches are updated continually as new builds are scheduled, and the caches on the nodes are refreshed every time the images from which they're booted are assembled (typically daily unless something has temporarily broken our ability to rebuild a particular image). So on average, the executor is pushing only 12 hours worth of new commits for each required project. I don't recall if it performs those push operations in parallel, but I suppose that's something we could look into. -- Jeremy Stanley