Once you said that, I looked through the actual code of the prepare-workspace-git role more carefully and you're right - all actions are made against already cached repos there. However since it mostly uses commands, it would still be the way more efficient to make up some module to replace all commands/shell to run things in multiprocess way.

Regarding example, you can take any random task from osa, ie [1] - it takes a bit more then 6 mins. When load on providers is high (or their volume backend io is poor), time increases

[1] https://zuul.opendev.org/t/openstack/build/52983abbcf8947f8926af0b1c07ef855/log/job-output.txt#176

05.02.2021, 00:58, "Jeremy Stanley" <fungi@yuggoth.org>:

On 2021-02-04 22:29:35 +0200 (+0200), Dmitriy Rabotyagov wrote:
For OSA huge issue is how zuul clones required-projects. Just this
single action takes for us from 6 to 10 minutes.
[...]

I'd be curious to see some examples of this. Zuul doesn't clone
required-projects, but it does push new commits from a cache on the
executor to a cache on the node. The executor side caches are
updated continually as new builds are scheduled, and the caches on
the nodes are refreshed every time the images from which they're
booted are assembled (typically daily unless something has
temporarily broken our ability to rebuild a particular image). So on
average, the executor is pushing only 12 hours worth of new commits
for each required project. I don't recall if it performs those push
operations in parallel, but I suppose that's something we could look
into.
--
Jeremy Stanley

--
Kind Regards,

Dmitriy Rabotyagov