Open Stack

On 02/08/2017 01:28 PM, Jeremy Stanley wrote:
> On 2017-02-08 14:10:16 -0500 (-0500), Paul Belanger wrote:
> [...]
>> We basically create a publisher in JJB to handle this, which looks in the
>> $WORKSPACE/images folder for things to scp to tarballs.o.o.  Take a look at the
>> kolla job for example.
>>
>> What we can do, is create an experimental job first test the uploads to
>> tarballs.o.o, once working, we can then update the jobs you have listed.
> [...]
>
> Friendly reminder to also keep an eye on
> http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=311&rra_id=all
> since its current flavor tops out at 800mbps and can cause some
> nasty blowback in the CI if it gets pegged for extended periods of
> time. Longer-term, we need a way to publish artifacts our jobs are
> reusing into the AFS mirror network so that the traffic can be
> localized to each provider and not, e.g., dragged across the
> Atlantic Ocean when jobs are running on a node in France.
>

We should be okay on that for tripleo-ci since we have a local squid 
proxy running in our cloud that our jobs are configured to use.  We'd 
want to make sure it was getting used for these images if we moved them 
out of the cloud, but it shouldn't drive crazy amounts of bandwidth as 
long as things are configured correctly.

Otherwise I'm fine with moving to tarballs.  The one thing our cloud is 
pretty short on is storage space - most of the compute nodes are 200 GB 
SSDs, which are fine for small-ish test vms, but not so good for big 
storage vms.

That being said, we were pretty cavalier with the disk on our current 
mirror.  We're mirroring a bunch of CentOS images we don't even use, and 
we upload 3 copies of the images but only use one.  There's a lot we 
need to clean up about this process regardless of where the images are 
stored.