Open Stack

On 2014-08-11 08:07:02 -0400 (-0400), Russell Bryant wrote:
> There's a pypi mirror at least.  I'm not sure about mirroring
> other things.

Right, that's a specific solution for mirroring the pypi.python.org
cheeseshop. We've got our (Infra) sights set on mirroring Ubuntu and
CentOS package repositories to similarly reduce the incidence of job
run-time failures we see getting updated packages and indexes from
distro sites and cloud provider mirrors, but those too are
nontrivial efforts and will be a while yet before we have them in
place. Mirroring things is generally complex, since different kinds
of files/data need widely differing retrieval, indexing and caching
solutions--there's no one-size-fits-all option really.

Perhaps another good example is the Fedora qcow2 image we download
and cache on DevStack worker images so that Heat can perform some of
its more complex integration tests... failures encountered when
obtaining that image from dl.fedoraproject.org are (last time I
checked anyway) our most frequent cause of nodepool update problems.
We could set up our own mirror of that file of course, but to some
extent that's still just moving the problem--each additional
mirroring solution is something new we have to monitor, maintain and
troubleshoot so we must ask ourselves whether the increased
management burden from that new complexity is balanced by potential
decreases in management burden found by improving stability in other
parts of the system.
-- 
Jeremy Stanley