Open Stack

On 8/6/19 10:49 AM, Clark Boylan wrote:
> On Tue, Aug 6, 2019, at 8:26 AM, Ben Nemec wrote:
>> Just a reminder that there is also
>> http://lists.openstack.org/pipermail/openstack-dev/2016-April/092546.html
>> which was intended to address this same issue.
>>
>> I toyed around with it a bit for TripleO installs back then and it did
>> seem to speed things up, but at the time there was a bug in our client
>> plugin where it was triggering a prompt for input that was problematic
>> with the server running in the background. I never really got back to it
>> once that was fixed. :-/
> 
> I'm not tied to any particular implementation. Mostly I wanted to show that we can take this ~5 minute portion of devstack and turn it into a 15 second portion of devstack by improving our use of the service APIs (and possibly even further if we apply it to all of the api interaction). Any idea how difficult it would be to get your client as a service stuff running in devstack again?

I wish I could take credit, but this is actually Dan Berrange's work. :-)

> 
> I do not think we should make a one off change like I've done in my POC. That will just end up being harder to understand and debug in the future since it will be different than all of the other API interaction. I like the idea of a manifest or feeding a longer lived process api update commands as we can then avoid requesting new tokens as well as pkg_resource startup time. Such a system could be used by all of devstack as well (avoiding the "this bit is special" problem).
> 
> Is there any interest from the QA team in committing to an approach and working to do a conversion? I don't want to commit any more time to this myself unless there is strong interest in getting changes merged (as I expect it will be a slow process weeding out places where we've made bad assumptions particularly around plugins).
> 
> One of the things I found was that using names with osc results in name to id lookups as well. We can avoid these entirely if we remember name to id mappings instead (which my POC does). Any idea if your osc as a service tool does or can do that? Probably have to be more careful for scoping things in a tool like that as it may be reused by people with name collisions across projects/users/groups/domains.

I don't believe this would handle name to id mapping. It's a very thin 
wrapper around the regular client code that just makes it persistent so 
we don't pay the startup costs every call. On the plus side that means 
it basically works like the vanilla client, on the minus side that means 
it may not provide as much improvement as a more targeted solution.

IIRC it's pretty easy to use, so I can try it out again and make sure it 
still works and still provides a performance benefit.

> 
>>
>> On 7/26/19 6:53 PM, Clark Boylan wrote:
>>> Today I have been digging into devstack runtime costs to help Donny Davis understand why tempest jobs sometimes timeout on the FortNebula cloud. One thing I discovered was that the keystone user, group, project, role, and domain setup [0] can take many minutes [1][2] (in the examples here almost 5).
>>>
>>> I've rewritten create_keystone_accounts to be a python tool [3] and get the runtime for that subset of setup from ~100s to ~9s [4].  I imagine that if we applied this to the other create_X_accounts functions we would see similar results.
>>>
>>> I think this is so much faster because we avoid repeated costs in openstack client including: python process startup, pkg_resource disk scanning to find entrypoints, and needing to convert names to IDs via the API every time osc is run. Given my change shows this can be so much quicker is there any interest in modifying devstack to be faster here? And if so what do we think an appropriate approach would be?
>>>
>>> [0] https://opendev.org/openstack/devstack/src/commit/6aeaceb0c4ef078d028fb6605cac2a37444097d8/stack.sh#L1146-L1161
>>> [1] http://logs.openstack.org/05/672805/4/check/tempest-full/14f3211/job-output.txt.gz#_2019-07-26_12_31_04_488228
>>> [2] http://logs.openstack.org/05/672805/4/check/tempest-full/14f3211/job-output.txt.gz#_2019-07-26_12_35_53_445059
>>> [3] https://review.opendev.org/#/c/673108/
>>> [4] http://logs.openstack.org/08/673108/6/check/devstack-xenial/a4107d0/job-output.txt.gz#_2019-07-26_23_18_37_211013
>>>
>>> Note the jobs compared above all ran on rax-dfw.
>>>
>>> Clark
>>>
>>
>>
>