Open Stack

For log storage I would definitely start with compression since these are
just plain text. Make sure you enable gzip decompression in your web server
software so people can still view the log files in their browser.

Before spending tons of disk space on log storage, I would also have it
purge logs from successful runs that are more than a few weeks old. The
ones people are interested in are the ones that break.
On Jul 10, 2014 1:08 AM, "Luke Gorrie" <luke at snabb.co> wrote:

> Howdy!
>
> I've been operating a shellci for a while now and overall it is very
> smooth.
>
> The main new feature now is to automatically retrigger events that neither
> definitely succeed (exit status 100) nor definitely fail (exit status 101).
> In this case the CI will vote "0" with the logs and then automatically
> schedule a new test, up to a max of 5 tests, until it can conclude +1 or -1.
>
> This seems to work well. On my setup I often see tests fail due to network
> problems (e.g. timeout during a Git checkout) and automatic retries filer
> away this class of error very neatly, ultimately casting the +1/-1 votes
> based purely on the relatively reliable contents of tempest.log. (Thanks
> Jim Gray for pointing out that detectable intermittent bugs are really easy
> to deal with http://www.hpl.hp.com/techreports/tandem/TR-85.7.pdf).
>
> [The network problems themselves are a little suspicious but for now I see
> them as a blessing that helps me make shellci robust and I don't worry too
> much about the root cause.]
>
> I'm running 5 parallel builds at the moment. I've tried ramping that up to
> 10 or 20 but then I start to see Vagrant/VirtualBox startup errors. Again,
> I'm using this as a test case for the automatic retries, and not digging
> too deep yet.
>
> I'd like to be able to track the results via RSS feeds. Can I do this via
> openstack.org or should I support that directly in shellci?
>
> disk space will become an issue soon. Currently I'm running around 500
> builds per day and each one generates around 8MB of logs. Compression and
> redundancy-avoidance may go a long way towards reducing his problem,
> however it seems a pity to have a "scarcity" mentality when it comes to
> logging (I'd prefer to do more rather than less). Any bright ideas for
> conveniently archiving some terabytes of logs?
>
> Overall I'd say that shellci is quite reliable now. The main reason I
> don't recommend it to others yet is that I'm a bit too busy to provide
> support with setting it up this week.
>
> Cheers,
> -Luke
>
>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140710/c8da8409/attachment.html>