Open Stack

On Thu, Mar 30, 2017 at 11:01:08AM -0400, Paul Belanger wrote:
> On Thu, Mar 30, 2017 at 03:08:57PM +0100, Steven Hardy wrote:
> > To be fair, we discussed this on IRC yesterday, everyone agreed infra
> > supported docker cache/registry was a great idea, but you said there was no
> > known timeline for it actually getting done.
> > 
> > So while we all want to see that happen, and potentially help out with the
> > effort, we're also trying to mitigate the fact that work isn't done by
> > working around it in our OVB environment.
> > 
> > FWIW I think we absolutely need multinode container jobs, e.g using infra
> > resources, as that has worked out great for our puppet based CI, but we
> > really need to work out how to optimize the container download speed in
> > that environment before that will work well AFAIK.
> > 
> > You referenced https://review.openstack.org/#/c/447524/ in your other
> > reply, which AFAICS is a spec about publishing to dockerhub, which sounds
> > great, but we have the opposite problem, we need to consume those published
> > images during our CI runs, and currently downloading images takes too long.
> > So we ideally need some sort of local registry/pull-through-cache that
> > speeds up that process.
> > 
> > How can we move forward here, is there anyone on the infra side we can work
> > with to discuss further?
> > 
> Yes, I am currently working with clarkb to adress some of these concerns. Today
> we are looking at setup our cloud mirrors to cache[1] specific URLs, for example
> we are trying testing out http://trunk.rdoproject.org  This is not a long term
> solution for projects, but a short. It will be opt-in for now, rather then us
> set it up for all jobs.  Long term, we move rdoproject.org into AFS.
> 
> I have been trying to see if we can do the same for docker hub, and continue to
> run it.  The main issue, at least for me, is we don't want to depend on docker
> tooling for this. I'd rather not install a docker into our control play at this
> point in time.
> 
> So, all of that to stay, it will take some time. I understand it is a high
> priority, but lets solve the current mirroring issues with tripleo first (RDO,
> gems, github), and lets see if the apache cache proxy with work for
> hub.docker.com too.
> 
> [1] https://review.openstack.org/451554

Wanted to follow up to this thread, we managed to get a reverse proxy cache[2]
for https://registry-1.docker.io working. So far, I've just tested ubuntu,
fedora, centos images but the caching works. Once we land this, any jobs using
docker can take advantage of the mirror.

[2] https://review.openstack.org/#/c/453811