Open Stack

Hello,

For the past year or two, I've maintained some type of deployment script to
build an all-in-one OpenStack environment which would allow me to test app
code (in my specific case, Terraform and Gophercloud). I've gone through a
lot of iterations and hit a milestone today.

There was interest about this back in March (
http://lists.openstack.org/pipermail/openstack/2017-March/018965.html) so I
figured I'd provide an update as well as just share this in general.

Describing this might come off as a little disjointed, but bear with me.

This is all available here:

https://github.com/terraform-providers/terraform-provider-openstack/tree/master/test-infra/standard

Right now, there are only scripts to build and deploy to AWS, but I have
OpenStack-compatible scripts I will be adding soon.

The first step is to use Packer to build an AMI (and soon Glance) image:

https://github.com/terraform-providers/terraform-provider-openstack/tree/master/test-infra/standard/aws/packer

$ packer build packstack-standard.json

Packer will run this file:

https://github.com/terraform-providers/terraform-provider-openstack/blob/master/test-infra/standard/files/deploy.sh

which will trigger a Packstack installation of Ocata. It configures Cinder
to use the NFS driver (which nixes testing of snapshots, but I'll sort that
out later^1). It then creates some standard flavors and adds additional
info to the openrc files. Then it begins cleaning up various OpenStack
services. Finally, it drops an rc.local file:

https://github.com/terraform-providers/terraform-provider-openstack/blob/master/test-infra/standard/files/rc.local

The above is a one-time action.

Now you can launch this image and have a clean all-in-one environment
within 5-10 minutes. To do that, you can use something like Terraform:

https://github.com/terraform-providers/terraform-provider-openstack/blob/master/test-infra/standard/aws/main.tf

When an instance is created, rc.local is run which will update the Keystone
catalog with the public IP of the instance. It'll restart all services and
then copy the openrc files to /var/www/html.

Terraform will run wget in a loop trying to grab these files. Once it
succeeds, you know rc.local has finished and the instance is ready to go.
You can then source the files and run tests against the environment. You
can also configure the instance to have an SSH key so you can also check
out system logs for debugging failures.

Earlier I mentioned a milestone: this setup was integrated into Terraform's
main CI system today, which means a full suite of tests (^2) will be run
nightly rather than partial and ad-hoc (which is still useful when
targeting specific bugs/features).

If anyone has any suggestions on how to make this better, definitely let me
know!

Speaking with David Flanders, I realized there might be interest in
providing complementary build scripts to easily set up a Go dev environment
(whether for Gophercloud, Terraform, or both). I would be happy to put that
together.

Thanks,
Joe

^1: Fun fact: using the standard LVM driver on a Cinder volume also hosted
on LVM *and* having both call the volume-group "cinder-volumes" will break
things. This made me think about how safe running tests against production
OpenStack clouds might be (anything from triggering a stuck volume to not
cleaning up resources).

^2: The Terraform OpenStack provider currently tests 156 different
Terraform configurations. You can see what's being tested by browsing the
*_test.go files here:

https://github.com/terraform-providers/terraform-provider-openstack/tree/master/openstack

For example:

https://github.com/terraform-providers/terraform-provider-openstack/blob/master/openstack/resource_openstack_compute_instance_v2_test.go#L637-L1134

Whenever someone reports a Terraform bug, I make sure to add a test for it.
This is common sense for general development, but in the context of
infrastructure-as-code, testing against different infrastructure
configurations is pretty cool.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-api-consumers/attachments/20170711/aa2bef9d/attachment.html>