Unfortunately pagination was removed from keystone in the v3 api and as far as we're aware it was never re-added. https://docs.openstack.org/api-ref/identity/v3/?expanded=get-available-proje... https://docs.openstack.org/api-ref/identity/v3/?expanded=get-available-proje... The only scope to limit the returns from keystone is with "list_limit" in the keystone.conf, but that just means that any given api call will return no more than that set value. We're looking at skyline, but need SSO integrations added before we can use it (which is in our backlog for Sergey) ________________________________ From: Dmitriy Rabotyagov <noonedeadpunk@gmail.com> Sent: 17 July 2022 21:12 Cc: openstack-discuss <openstack-discuss@lists.openstack.org> Subject: Re: [dev] directions to the right project team CAUTION: This email originates from outside THG ________________________________ Just out of interest, have you tried using Skyline as a dashboard instead of the horizon? I believe keystone should already support pagination. The better question is what horizon uses as client - openstacksdk or python-keystoneclient. As I believe a lot of issues are solved with openstacksdk as of today, and switching to it also would match community goal. So imo that might be good thing to contribute to:) вс, 17 июл. 2022 г., 22:07 Sergey Drozdov <sergey.drozdov.dev@gmail.com<mailto:sergey.drozdov.dev@gmail.com>>: Too whom it may concern, I am relatively new to contributing to OpenStack and therefore am not sure who to direct this question too. I have the following issue. The firm I am currently working at is running OpenStack at sale with circa 6000 different projects. Whenever we try to access a projects tab via horizon we experience a timeout (accessing through the API works fine). To that end we were hoping to implement something akin to pagination or filtering. I was hoping to work on this and contribute the outcome to OpenStack. Is this something worthwhile and which group should I talk to; keystone, horizon or both ? I apologise in advance if I have missed something. Best Regards, Sergey Danny Webb Principal OpenStack Engineer The Hut Group<http://www.thehutgroup.com/> Tel: Email: Danny.Webb@thehutgroup.com<mailto:Danny.Webb@thehutgroup.com> For the purposes of this email, the "company" means The Hut Group Limited, a company registered in England and Wales (company number 6539496) whose registered office is at Fifth Floor, Voyager House, Chicago Avenue, Manchester Airport, M90 3DQ and/or any of its respective subsidiaries. Confidentiality Notice This e-mail is confidential and intended for the use of the named recipient only. If you are not the intended recipient please notify us by telephone immediately on +44(0)1606 811888 or return it to us by e-mail. Please then delete it from your system and note that any use, dissemination, forwarding, printing or copying is strictly prohibited. Any views or opinions are solely those of the author and do not necessarily represent those of the company. Encryptions and Viruses Please note that this e-mail and any attachments have not been encrypted. They may therefore be liable to be compromised. Please also note that it is your responsibility to scan this e-mail and any attachments for viruses. We do not, to the extent permitted by law, accept any liability (whether in contract, negligence or otherwise) for any virus infection and/or external compromise of security and/or confidentiality in relation to transmissions sent by e-mail. Monitoring Activity and use of the company's systems is monitored to secure its effective use and operation and for other lawful business purposes. Communications using these systems will also be monitored and may be recorded to secure effective use and operation and for other lawful business purposes. hgvyjuv

It was a conscious decision by the keystone team back in 2015 as far as we can tell. There was a large discussion on the mailing list regarding this that seemed to have left this issue unresolved (most specifically around user pagination, but this seems to have affected project / domain elements as well). https://lists.openstack.org/pipermail/openstack-dev/2015-August/thread.html#... Ultimately what we're seeking to do is start a discussion within the keystone / horizon (and potentially skyline) community about how we can rectify the current issues we're facing around the usability of the UX portion of keystone elements. Eg, if pagination isn't the way the keystone community wants to go should we look at instead having dynamic filtering instead in the UI? Are there any other options that people can think of that might be a better way forward? ________________________________ From: Julia Kreger <juliaashleykreger@gmail.com> Sent: 19 July 2022 14:21 To: Danny Webb <Danny.Webb@thehutgroup.com> Cc: Dmitriy Rabotyagov <noonedeadpunk@gmail.com>; openstack-discuss <openstack-discuss@lists.openstack.org> Subject: Re: [dev] directions to the right project team CAUTION: This email originates from outside THG On Tue, Jul 19, 2022 at 3:44 AM Danny Webb <Danny.Webb@thehutgroup.com> wrote:

Unfortunately pagination was removed from keystone in the v3 api and as far as we're aware it was never re-added.

This is quite concerning, and a quick look at the code confirms it. Mostly. There are remnants of "hints" and SQL Query filtering, but the internal limit is just a truncation which seems bad as well. https://github.com/openstack/keystone/blame/d7b1d57cae738183f8d85413e942402a8a4efb31/keystone/server/flask/common.py#L675<https://github.com/openstack/keystone/blame/d7b1d57cae738183f8d85413e942402a8a4efb31/keystone/server/flask/common.py#L675> This seems like a fundamental performance oriented feature because the overhead in data conversion can be quite a bit when you have a large number of well... any objects being returned from a database. Does anyone know if a bug is open for this issue? Danny Webb Principal OpenStack Engineer The Hut Group<http://www.thehutgroup.com/> Tel: Email: Danny.Webb@thehutgroup.com<mailto:Danny.Webb@thehutgroup.com> For the purposes of this email, the "company" means The Hut Group Limited, a company registered in England and Wales (company number 6539496) whose registered office is at Fifth Floor, Voyager House, Chicago Avenue, Manchester Airport, M90 3DQ and/or any of its respective subsidiaries. Confidentiality Notice This e-mail is confidential and intended for the use of the named recipient only. If you are not the intended recipient please notify us by telephone immediately on +44(0)1606 811888 or return it to us by e-mail. Please then delete it from your system and note that any use, dissemination, forwarding, printing or copying is strictly prohibited. Any views or opinions are solely those of the author and do not necessarily represent those of the company. Encryptions and Viruses Please note that this e-mail and any attachments have not been encrypted. They may therefore be liable to be compromised. Please also note that it is your responsibility to scan this e-mail and any attachments for viruses. We do not, to the extent permitted by law, accept any liability (whether in contract, negligence or otherwise) for any virus infection and/or external compromise of security and/or confidentiality in relation to transmissions sent by e-mail. Monitoring Activity and use of the company's systems is monitored to secure its effective use and operation and for other lawful business purposes. Communications using these systems will also be monitored and may be recorded to secure effective use and operation and for other lawful business purposes. hgvyjuv

1. Skyline UX support project pagination from the very beginning since we also meet this “too many keystone projects” issue in previous projects 2. https://bugs.launchpad.net/skyline-apiserver/+bug/1972736 skyline SSO support is on the way, & supposed could finish before August, 15th. Thanks Best Regards Wenxiang Wu From: <openstack-discuss-bounces+wu.wenxiang=99cloud.net@lists.openstack.org> on behalf of Danny Webb <Danny.Webb@thehutgroup.com> Date: Tuesday, July 19, 2022 at 21:49 To: Julia Kreger <juliaashleykreger@gmail.com> Cc: Dmitriy Rabotyagov <noonedeadpunk@gmail.com>, openstack-discuss <openstack-discuss@lists.openstack.org> Subject: Re: [dev] directions to the right project team It was a conscious decision by the keystone team back in 2015 as far as we can tell. There was a large discussion on the mailing list regarding this that seemed to have left this issue unresolved (most specifically around user pagination, but this seems to have affected project / domain elements as well). https://lists.openstack.org/pipermail/openstack-dev/2015-August/thread.html#... Ultimately what we're seeking to do is start a discussion within the keystone / horizon (and potentially skyline) community about how we can rectify the current issues we're facing around the usability of the UX portion of keystone elements. Eg, if pagination isn't the way the keystone community wants to go should we look at instead having dynamic filtering instead in the UI? Are there any other options that people can think of that might be a better way forward? From: Julia Kreger <juliaashleykreger@gmail.com> Sent: 19 July 2022 14:21 To: Danny Webb <Danny.Webb@thehutgroup.com> Cc: Dmitriy Rabotyagov <noonedeadpunk@gmail.com>; openstack-discuss <openstack-discuss@lists.openstack.org> Subject: Re: [dev] directions to the right project team CAUTION: This email originates from outside THG On Tue, Jul 19, 2022 at 3:44 AM Danny Webb <Danny.Webb@thehutgroup.com> wrote:

Unfortunately pagination was removed from keystone in the v3 api and as far as we're aware it was never re-added.

This is quite concerning, and a quick look at the code confirms it. Mostly. There are remnants of "hints" and SQL Query filtering, but the internal limit is just a truncation which seems bad as well. https://github.com/openstack/keystone/blame/d7b1d57cae738183f8d85413e942402a... This seems like a fundamental performance oriented feature because the overhead in data conversion can be quite a bit when you have a large number of well... any objects being returned from a database. Does anyone know if a bug is open for this issue? Danny Webb Principal OpenStack Engineer The Hut Group Tel: Email: Danny.Webb@thehutgroup.com For the purposes of this email, the "company" means The Hut Group Limited, a company registered in England and Wales (company number 6539496) whose registered office is at Fifth Floor, Voyager House, Chicago Avenue, Manchester Airport, M90 3DQ and/or any of its respective subsidiaries. Confidentiality Notice This e-mail is confidential and intended for the use of the named recipient only. If you are not the intended recipient please notify us by telephone immediately on +44(0)1606 811888 or return it to us by e-mail. Please then delete it from your system and note that any use, dissemination, forwarding, printing or copying is strictly prohibited. Any views or opinions are solely those of the author and do not necessarily represent those of the company. Encryptions and Viruses Please note that this e-mail and any attachments have not been encrypted. They may therefore be liable to be compromised. Please also note that it is your responsibility to scan this e-mail and any attachments for viruses. We do not, to the extent permitted by law, accept any liability (whether in contract, negligence or otherwise) for any virus infection and/or external compromise of security and/or confidentiality in relation to transmissions sent by e-mail. Monitoring Activity and use of the company's systems is monitored to secure its effective use and operation and for other lawful business purposes. Communications using these systems will also be monitored and may be recorded to secure effective use and operation and for other lawful business purposes. hgvyjuv