Open Stack

My workflow is very similar to Joe's but simpler because I only need
to model one could.

I have a base image that is ubuntu-cloud-image + my local config
management hooks and auth bits.  This is how most of my production VM
start out as well.  Our puppet setup uses local facts (via stdlib's
/etc/facter/facts.d/) to classify a host then hiera fills in the bits
including what classes to apply.  So if I echo
"role=openstack-controller \ncluster=openstack-test" to
/etc/facter/facts.d/some-file.txt using cloudinit and user-data then I
have a test version of my controller node.

In production we use http://fai-project.org where Joe uses cobbler, it
does a PXE install via nfsroot and has some detection-fu to set up the
hardware bits and facts.d file(s) then calls puppet at the end.

That much is fairly trivial, if you're playing this game you probably
already had lots of hardware and a way to deploy it before OpenStack &
we all have our favorites.  No harm in swapping stories but there lots
of right ways to skin that cat.


I guess what I'm most interested in is that while the configuration is
easy to replicate and test, in production the data accumulated through
the life of the production systems has a big impact on upgrades, or at
least can.  How can you model that?

Pulling in a dump of the production DB let me model the upgrade
failure I experienced in production and find three more similar points
that broke down with the production database but worked fine with a
clean config.  Trouble is that pulls in many assumptions about the
production system that are untrue in the test environment, it can't
talk to those compute-nodes, it doesn't really have those
glance-images, it's not authorized to talk to that SAN server, and
more over I don't want it to be.

I'm probably going to end up doing two passes of testing for upgrade,
unless I get a flash of inspiration.  First testing that the endstate
is want I want using methods similar to what people are talking about
to model configuration.  Second modeling the database upgrade
migration using a dump of the production database to see if the db
sync's apply cleanly.  My goal of course would be to do this in one
pass, but not sure I can both have the production DB and test
environment that will function without really fully duplicating
everything which is obviously too damned expensive.

-Jon