[Openstack-operators] How do you keep your test env sync'ed to production?

Joe Topjian joe at topjian.net
Thu Jan 9 18:08:37 UTC 2014

Hi Jay,

(quick note: I have not had a chance to test a patched neutron environment)

This sounds virtually identical to the Triple-O project's architecture
> of an undercloud and an overcloud -- and it makes sense, since a driving
> factor behind Triple-O is continuous deployment testing. :)

This seems to be the word on the street  :)

> I'm curious, though... how often do you change/update the *undercloud*
> -- the OpenStack installation that deploys the bare-metal machines? Is
> it just the *overcloud* that follows the master branches, or do you
> update the undercloud at a similar pace?

The undercloud's configuration is also in Puppet, but it rarely changes.
9/10 times the undercloud is refreshed to demo production bare-metal
deployments to new staff. The config changed once in 2013 and that was to
move from Grizzly to Havana.

> Are you using Neutron? If so, what setup are you using for the
> undercloud and the overcloud? Are you using a flat or Vlan model for the
> undercloud and an SDN model (GRE? VxLAN?) for the overcloud?

At the moment, the undercloud is still nova-network and VlanManager. It is,
however, using Open vSwitch with brcompat.

I found that with the standard bridge module, nested instances (instances
running in the overcloud) were not able to get IPs via DHCP. Somewhere
along the path, the packet was just being dropped or ignored. Using
ovs+brcompat resolved that issue.

I tried doing a simple Neutron setup on the undercloud earlier this year,
but ran into this issue:


I haven't re-tried since I posted that message.

> LOL. We had the same issue in our testing environments at AT&T. :)
> Strangely, it was difficult to find a 3270 filer lying around that
> nobody wanted ;) So, instead we just tested on the iSCSI basic drivers
> for Cinder.

Yup, that's what we do, too.

> Interesting. Did you ever investigate using LXC containers instead of
> KVM/Qemu VMs for your testing environment OpenStack instances?

I recently started looking into LXC as a compute driver in the past month,
but just out of personal interest. I'm not sure if I want to use it to
emulate our production environment, though. qemu might be dog slow, but
it's still KVM which is what the production environments use.

> ++ You may want to check with Mikal Still (cc'd) about linking into
> turbo-hipster for testing schema migrations on "realistic" databases,
> too.

Oh thanks! I forgot about turbo-hipster. I now remember the release
announcement + when Mikal was requesting production dbs to donate.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20140109/00022594/attachment.html>

More information about the OpenStack-operators mailing list