On Tue, Jul 26, 2022 at 8:56 PM Stephen Finucane <stephenfin@redhat.com> wrote:
On Mon, 2022-07-25 at 18:20 +0100, Sergey Drozdov wrote:
To whom it may concern,
I have previously sen out an email about this topic but just wanted to run this by community again incase it was missed.
I have the following issue. The firm I am currently working at is running OpenStack at sale with circa 6000 different projects. Whenever we try to access a projects tab via horizon we experience a timeout (accessing through the API works fine).
For the horizon team, we were hoping to propose something akin to filtering/dynamic listing in order to rectify the issue.
I suspect you'll have a difficult time getting this into Horizon since I'm unsure how many people are actively working on the project. Something AJAX'y whereby you only load the first N results and insist on more filtering (or use of the API) for anything more would be a good start?
I am not very active in horizon like before right now, so I waited for others reply the thread :p I talked with Sergey on #-horizon IRC channel. As long as keystone does not support paginations for projects (and other resources), perhaps a way to mitigate the issue discussed here would be that horizon emulates the pagination for projects. Horizon fetches all projects from keystone API and horizon server side support pagination by narrowing into a subset of projects. The default implementation of the "Projects" panel is based on Django and the keystone API itself works well with many projects like 6000 projects (according to this thread), so it looks like the bottleneck would be during handling projects inside horizon, perhaps during rendering the project tables. Anyway, it would be nice if keystone supports pagination for projects of course.
For the keystone team, we were wondering whether there is anything we can do with the API in order to simplify the aforementioned horizon proposal; we are hoping to get both teams on board.
Based on Julia's reply, it seems pagination for the "users" API might still be a non-starter. You could probably look to implement it for other resources like projects and domains though. Once available in Keystone, it should be pretty easy to add to Horizon (pagination is available elsewhere). I know Keystone is undergoing a bit of a revival right now so there's a chance this will get more traction...though of course you'll still need to add stuff to Horizon at a later date.
It is good to hear that a kind of revival of keystone. How is it likely that keystone pagination support happens? AFAIK, keystone dropped the pagination support as it is not easy to support pagination for some identity backends, so I am wondering there is a magic to support pagination well soon. if it happens soon, it is better for horizon to wait for the keystone pagination support again.
We are ready to open up a bug report and start subsequent development should this be supported by both teams!
Horizon uses blueprints, as discussed in the Horizon contributors guide [1]. Creating an initial blueprint and some PoC patches might be a good start? Keystone uses blueprints and specs [2][3] so if you want to extend this then you'll probably need to draft a spec then the PoC patches to prove out the idea.
Mainly from horizon perspective, the most challenging is how to coordinate the effort between keystone and horizon like whether keystone has a plan to support pagination again. The direction in horizon totally depends on decisions in keystone. If it does not happen soon in keystone, horizon needs to explore a way to improve the performance without keystone pagination support. Thanks, Akihiro Motoki (amotoki)
Stephen
[1] https://docs.openstack.org/horizon/latest/contributor/contributing.html [2] https://docs.openstack.org/keystone/latest/contributor/contributing.html [3] https://opendev.org/openstack/keystone-specs
I am available on the both Openstack-horizon and Openstack-keystone irc channels under the nickname sdrozdov, please feel free to reach out. Best Regards, Sergey Drozdov Software Engineer The Hut Group