[openstack-dev] [horizon] Fetch resources in parallel

Ivan Kolodyazhny e0ne at e0ne.info
Wed Dec 6 10:47:48 UTC 2017

Hi Team,

Last week we had a hot discussion about this topic at the meeting [6]. We
still don't have an agreement how to go forward with it. As was requested,
I did some perf testing with a current approach [3]. TBH, these tests
results don't show real data but look realistic. I used the same VM with
the same code (just checked out to the previous commit of horizon). To
get better results, we need to re-test all but I don't have enough time
now:(. I hope, these results are good enoudh and we can make our final
decision on this topic.


Ivan Kolodyazhny,

On Tue, Oct 31, 2017 at 11:06 PM, Ivan Kolodyazhny <e0ne at e0ne.info> wrote:

> Just forgot to mention one other option: we can use celery [6] too but I
> didn't do any performance testing with it. This solution will be more
> complex.
> [6] http://www.celeryproject.org/
> Regards,
> Ivan Kolodyazhny,
> http://blog.e0ne.info/
> On Tue, Oct 31, 2017 at 7:15 PM, Ivan Kolodyazhny <e0ne at e0ne.info> wrote:
>> Hi team,
>> We all know that unfortunately Horizon's performance not good enough in
>> some cases. That's why we try to fix it on both sides: client-side and
>> server-side.
>> I would like to talk mostly about server-side now. On some views, we use
>> 'futurist' library [1] to fetch resources from API's in parallel. I've
>> filed a blueprint for this effort [2]. You can find some work in progress
>> patches in the Gerrit.
>> For now, we use futurist.ThreadPoolExecutor in Horizon to fetch resources
>> from APIs. It requires some specific Apache configuration changes to allow
>> WSGI app to create threads. It means, our code execution depends on Apache
>> mod_wsgi configuration.
>> To avoid this, we can use futurist.GreenThreadPoolExecutor. It
>> uses eventlet green threads which can be used to speed-up I/O operations
>> like 'call external API'.
>> I did very simple performance testing [3] with proposed patch [4] for
>> volumes views. My tests are not ideal but you can see how it's going with
>> green thread. I propose to switch to the futurist.GreenThreadPoolExecutor
>> and add 'eventlet.monkey_patch()' call to the wsgi app for Horizon. It will
>> speed up parallel API calls and make Horizon independent on Apache or other
>> web server configuration.
>> I added this topic to the next weekly meeting agenda [5] so we can
>> discuss it there too.
>> [1] https://github.com/openstack/futurist/
>> [2] https://blueprints.launchpad.net/horizon/+spec/fetch-
>> resources-in-parallel
>> [3] https://docs.google.com/spreadsheets/d/14zDpdkPUfGDR_ZycaGUo
>> T64kHsGi1gL9JOha8n5PVeg/edit?usp=sharing
>> [4] https://review.openstack.org/#/c/426493/
>> [5] https://wiki.openstack.org/wiki/Meetings/Horizon#Agenda_
>> for_Next_Meeting
>> Regards,
>> Ivan Kolodyazhny,
>> http://blog.e0ne.info/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20171206/c7185dc7/attachment.html>

More information about the OpenStack-dev mailing list