Open Stack

On 07/24/2014 02:51 PM, Joshua Harlow wrote:
> A potentially brilliant idea ;-)
> 
> Aren't all the machines the gate runs tests on VMs running via OpenStack APIs?
> 
> OpenStack supports snapshotting (last time I checked). So instead of providing back a whole bunch of log files, provide back a snapshot of the machine/s that ran the tests; let person who wants to download that snapshot download it (and then they can boot it up into virtualbox, qemu, there own OpenStack cloud...) and investigate all the log files they desire. 
> 
> Are we really being so conservative on space that we couldn't do this? I find it hard to believe that space is a concern for anything anymore (if it really matters store the snapshots in ceph, or glusterfs, swift, or something else... which should dedup the blocks). This is pretty common with how people use snapshots and what they back them with anyway so it would be nice if infra exposed the same thing...
> 
> Would something like that be possible? I'm not so familiar with all the inner workings of the infra project; but if it eventually boots VMs using an OpenStack cloud, it would seem reasonable that it could provide the same mechanisms we are all already used to using...
> 
> Thoughts?

There are actual space concerns. Especially when we're talking about 20k
runs / week. At which point snapshots are probably in the neighborhood
of 10G, so we're talking about 200 TB / week of storage. Plus there are
actual technical details of the fact that glance end points are really
quite beta in the clouds we use. Remember our tests runs aren't pets,
they are cattle, we need to figure out the right distillation of data
and move on, as there isn't enough space or time to keep everything around.

Also portability of system images is... limited between hypervisors.

If this is something you'd like to see if you could figure out the hard
parts of, I invite you to dive in on the infra side. It's very easy to
say it's easy. :) Actually coming up with a workable solution requires a
ton more time and energy.

	-Sean

-- 
Sean Dague
http://dague.net