Open Stack

While I was working on the grenade refactor I was considering using
openstack client for some resource create / testing. Doing so made me
realize that osc is sluggish. From what I can tell due to the way it
loads the world, there is a minimum 1.5s overhead on every command
execution. For instance, "openstack server list" takes a solid extra
second over "nova list" in my environment.

I wrote a little tool to figure out how much time we're spending in
openstack client - https://review.openstack.org/#/c/172713/

On a randomly selected dsvm-full run from master it's about 4.5 minutes.
Now, that being side, there are a bunch of REST calls it's making, so
it's not all OSC's fault. However there is a lot of time lost to that
reload the world issue. Especially when we are making accounts.

For instance, the create accounts section of Keystone setup:
https://github.com/openstack-dev/devstack/blob/master/stack.sh#L968-L1016

Now takes 3.5 minutes in master -
http://logs.openstack.org/13/172713/1/check/check-tempest-dsvm-full/d3b0b8e/logs/devstacklog.txt.gz

2015-04-12 12:37:40.997 | + echo_summary 'Starting Keystone'
2015-04-12 12:41:06.833 | + echo_summary 'Configuring and starting Horizon'

The same chunk in Icehouse took just over 1 minute -
http://logs.openstack.org/28/165928/2/check/check-tempest-dsvm-full/f0b3e07/logs/devstacklog.txt.gz

2015-04-10 15:59:08.699 | + echo_summary 'Starting Keystone'
2015-04-10 16:00:00.313 | + echo_summary 'Configuring and starting Horizon'

In master we do create a few more accounts as well, again, it's not all
OSC, however OSC is definitely adding to it.

A really great comparison between OSC and Keystone commands is provided
by the ec2 user creation:

Icehouse:
http://logs.openstack.org/28/165928/2/check/check-tempest-dsvm-full/f0b3e07/logs/devstacklog.txt.gz#_2015-04-10_16_01_07_148

Master:
http://logs.openstack.org/13/172713/1/check/check-tempest-dsvm-full/d3b0b8e/logs/devstacklog.txt.gz#_2015-04-12_12_43_19_655

The keystone version of the commands take ~ 500ms, the OSC versions 1700ms.


So, under the current model I think we're paying a pretty high strategy
tax in OSC use in devstack. It's adding minutes of time in a normal run.
I don't know all the internals of OSC and what can be done to make it
better. But I think that as a CLI we should be as responsive as
possible. < 1s seems like it should be target for at least all the
keystone operations. I do think this is one of the places (like
rootwrap) where load time is something to not ignore.

	-Sean

-- 
Sean Dague
http://dague.net