Open Stack

On 2015-06-01 14:55:06 +0200 (+0200), Thomas Goirand wrote:
[...]
> So, should I start writing a script to build an image for package
> building (ie: an image with sbuild, git-buildpackage, and so on...)?
[...]

Probably what we'd want to do is something like debootstrap/rpmstrap
a chroot for each platform we want to build, then in each of them
iterate through the packaging git repos and --download-only the
build-deps listed therein. That will prime a local cache in each
chroot and then it will get baked into that image. Later when a
worker is booted from that image, the package build job just chroots
into the appropriate filesystem subtree and has a warm cache
available to it so it only needs to hopefully update at most the
package list and maybe a handful of packages before starting to
build whatever new package was requested.

The good thing about this approach is that it can be added later, it
doesn't need to be implemented on day one. If we design the jobs
correctly they'll download whatever they need, and then we can just
iterate on improving our caches to improve performance.

> How would a job get the latest version of a Git repository then? This
> still needs network, right?
[...]

The way our jobs work right now is that the workers start with a
recent (generally no more than a day old) clone of all the Git
repositories we maintain. It still has to hit the network to
retrieve more recent Git refs, but this at least minimizes network
interaction and significantly reduces the failure rate.
-- 
Jeremy Stanley