Open Stack

Hey folks!

So I've been working on some patches to speed up plan operations in
TripleO.  This was originally driven by the UI needing to be able to
perform a 'plan upload' in something less than several minutes. :)

https://review.openstack.org/#/c/581153/
https://review.openstack.org/#/c/581141/

I have a functioning set of patches, and it actually cuts over 2 minutes
off the overcloud deployment time.

Without patch:
+ openstack overcloud plan create --templates
/home/stack/tripleo-heat-templates/ overcloud
Creating Swift container to store the plan
Creating plan from template files in: /home/stack/tripleo-heat-templates/
Plan created.
real    3m3.415s

With patch:
+ openstack overcloud plan create --templates
/home/stack/tripleo-heat-templates/ overcloud
Creating Swift container to store the plan
Creating plan from template files in: /home/stack/tripleo-heat-templates/
Plan created.
real    0m44.694s

This is on VMs.  On real hardware it now takes something like 15-20 seconds
to do the plan upload which is much more manageable from the UI standpoint.

Some things about what this patch does:

- It makes use of process-templates.py (written for the undercloud) to
process the jinjafied templates.  This reduces replication with the
existing version in the code base and is very fast as it's all done on
local disk.
- It stores the bulk of the templates as a tarball in swift.  Any
individual files in swift take precedence over the contents of the tarball
so it should be backwards compatible.  This is a great speed up as we're
not accessing a lot of individual files in swift.

There's still some work to do; cleaning up and fixing the unit tests,
testing upgrades etc.  I just wanted to get some feedback on the general
idea and hopefully some reviews and/or help - especially with the unit test
stuff.

Thanks everyone!

    Ian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20180801/58307185/attachment.html>