Open Stack

Monty Taylor <mordred at inaugust.com> writes:

> As an added bonus - paste and etherpad should cease having any local
> data once the db is moved off - so we should be able to spin up more
> than one of them and put them behind a load balancer.

This seems like a good idea, and I'm looking forward to it.

> This isn't super urgent - but it sprung out of the "make sure we're
> backing up MySQL databases" bug - which then seemed like something we
> should, in fact, _not_ be dealing with.

I'm not sure what you're saying here, but I very strongly believe that
we should be dealing with database backups.  I really don't want to lose
the data we've created in the wiki, gerrit, etherpads, etc., and backups
are necessary for more than simple technical failures.

To recap the design goals for our backups:

 * recover data if we accidentally delete it due to operational error
 * have historic data if we discover a long running problem
 * audit trail in case of compromise (again, with historic data which
   can be restored from before a compromise)
 * be protected against accidental removal of, damage to, or loss of
   access to the cloud account where our servers are run

To that end, our design has write-only delta-based backups sent to at
least two servers on different cloud providers.

I'm looking forward to your implementing what you have described, but I
feel that having solid backups is extremely important, and I would like
to avoid making our database systems more complicated without doing
that.  So I think if you would like to proceed with this, it should
include a fully thought out and working backup system from the
beginning.

I believe, as we discussed earlier, the only thing missing from our
current backup system is little more than a simple mysqldump script that
runs immediately before the backups run.  I think that would probably
work fine as part of the system you describe above as well.  In the mean
time, would you consider adding that to our current systems so we can
start backing up our databases while you do some of the experimental
work you described above?

-Jim