[qa][openstackclient] Debugging devstack slowness

Clark Boylan cboylan at sapwetik.org
Wed Aug 14 17:20:37 UTC 2019


On Wed, Aug 14, 2019, at 10:07 AM, Ben Nemec wrote:
> I have a PoC patch up in devstack[0] to start using the openstack-server 
> client. It passed the basic devstack test and looking through the logs 
> you can see that openstack calls are now completing in fractions of a 
> second as opposed to 2.5 to 3, so I think it's working as intended.
> 
> That said, it needs quite a bit of refinement. For example, I think we 
> should disable this on any OSC patches. I also suspect it will fall over 
> for any projects that use an OSC plugin since the server is started 
> before any plugins are installed. This could probably be worked around 
> by restarting the service after a project is installed, but it's 
> something that needs to be dealt with.
> 
> Before I start taking a serious look at those things, do we want to 
> pursue this? It does add some potential complexity to debugging if a 
> client call fails or if the server crashes. I'm not sure I can quantify 
> the risk there though since it's always Just Worked(tm) for me.

Considering that our number one identified e-r bug is job timeouts [1] I think anything to reduce job time by measurable amounts is worthwhile. Additionally if we save 5 minutes per devstack run and then run devstack 10k times a day (not an up to date number but has been in that range in the past, someone can double check this with grafana or logstash or zuul dashboard) that is a massive savings when looked at on the whole. To me that makes it worthwhile.

> 
> -Ben
> 
> 0: https://review.opendev.org/676016
>
[1] http://status.openstack.org/elastic-recheck/index.html#1686542



More information about the openstack-discuss mailing list