Open Stack

Dear all,

In the need of a "lab TripleO" in order to validate updates before
pushing to production, I created some ansible receipt, and a whole
isolated network. This isolated network mimic 1:1 the current
production, and is mainly based on libvirt.

I named this "project" DoubleNO³, and it tells a lot of thing regarding
the concepts and architecture:
Double-NATed-tripleO

Some explanations:

On a dedicated baremetal, I've installed:
- CentOS 7
- Libvirt components

The main VMs are:
- virtualized RedHat Satellite (in order to use repository snapshots)
- virtualized undercloud (prod)
- virtualized undercloud (lab)
- virtualized computes (lab, 4 nodes)
- virtualized controllers (lab, 3 nodes)
- virtualized ceph-storages (lab, 2 nodes, with additional volumes for
Ceph setup emulation)

The lab instances share a network bridge (name it lab-trunk), and each
VM has the "right" amount of network interfaces (we use bonding in prod).

In order to isolate the lab, I've created a second layer, with a
dedicated "prenat" instance (lab-prenat). This one has two interfaces:
- one on a libvirt "NAT" network (eth0)
- one on lab-trunk bridge (eth1)

Regarding IPs, eth0 has a private IP in the 192.168.x.y scope, is the
default route to Internet via the libvirt NAT.
Eth1 has multiple IPs:
- one that emulates the public gateway of our production deploy
- all the IPMI addresses of our productive nodes

The second pool allows to keep Ironic configuration 1:1 with production.
In order to make IPMI calls, I've deployed VirtualBMC on the lab-prenat
instance, and am using the qemu+ssh capability of libvirt in order to
talk to the hypervisor and manage the VMs.

All of that is working pretty nicely together. As I decided to use a
virtual Undercloud node instead of baremetal, it was pretty easy to
duplicate the current undercloud node, drop the stack, and redeploy the
virtual lab in a swift way.

Of course, all wasn't as painless as it seems, I had "some" issues with
the network, especially for IPMI: I wanted to have the virtualBMC
directly on the hypervisor, and had many issues with libvirt itpables
rules. In the end, using the qemu+ssh was just the right, easy way to do
things. And this prevent any IPMI listener to be displayed on the outside.

But in the end: we actually have a 1:1 matching lab we can use in order
to validate every updates, and an easy way to rollback to a previous
consistent state using libvirt snapshots.

Would anyone be interested in a more "academic" documentation about the
whole stuff (especially the lab itself - satellite is an (interesting)
option)?
If so, where should I push that doc? Probably somewhere in "advanced
deployment" or something like that.

Unfortunately, I don't think I'll be able to open the ansible code soon,
but at least the concepts can be explained, and configuration example
provided.


Cheers,

C.

-- 
Cédric Jeanneret
Senior Linux System Administrator
Infrastructure Solutions

Camptocamp SA
PSE-A / EPFL, 1015 Lausanne

Phone: +41 21 619 10 32
Office: +41 21 619 10 02
Email: cedric.jeanneret at camptocamp.com

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 878 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20180207/9e0aa602/attachment.sig>