slow server list
Eugen Block
eblock at nde.ag
Wed Feb 8 10:25:00 UTC 2023
Hi,
the mysql approach is much faster, of course. But it also could be a
memcached issue if the requests take too long. In a customer cluster
with more than 400 VMs I get a response in under 7 seconds, 6 seconds
for the long nova request:
| GET https://control:8774/v2.1/servers/detail?all_tenants=True |
6.023134 |
| Total |
6.978378 |
In a different cluster with around 200 VMs the request comes back
after less than 4 seconds:
| GET http://controller:8774/v2.1/servers/detail?all_tenants=True |
2.263082 |
| Total |
3.6819349999999997 |
It appears to be proportional, so in a cluster with more than 1000 VMs
it seems reasonable to assume that it could take that long.
The worst thing we had was a few years back when listing all VMs from
all projects via dashboard timed out after one minute. That's when we
took the time to analyze and noticed a misconfiguration of memcached.
But I'm not sure how much room for improvement there is with so many
VMs, maybe other operators can report their experiences as well.
Regards,
Eugen
Zitat von Satish Patel <satish.txt at gmail.com>:
> I am experiencing the same behavior on all of my clouds. It's normal I
> believe because behind the command so many api calls go out to construct
> data. We have 500 vms in single projects so I wrote my own wrapper script
> to talk to mariadb and fetch that information directly from DB. This is not
> the best solution but works for me and now i can pull out any records
> within microseconds.
>
> $ mysql -h 172.28.15.79 -u user1 --password=<PASSWORD> nova -e "select
> display_name as
> hostname,vm_state,node,availability_zone,vcpus,memory_mb,root_gb from
> instances WHERE (vm_state='active' OR vm_state='stopped');"
>
>
>
> On Tue, Feb 7, 2023 at 2:07 PM Parsa Aminian <p.aminian.server at gmail.com>
> wrote:
>
>> hello
>> On openstack wallaby version openstack server list result too slow :
>> | GET http://example.com:35357/v3
>> | 0.040385 |
>> | POST http://example.com:35357/v3/auth/tokens
>> | 0.49567 |
>> | GET http://example.com:9292/
>> | 0.011694 |
>> | GET http://example.com:8774/v2.1/servers/detail?all_tenants=True
>> | 16.032512 |
>> | GET http://example.com:9292/v2/images
>> | 0.241679 |
>> | GET
>> http://example.com:9292/v2/images?marker=b17469eb-4548-4fbb-b871-5880323a7a76
>> | 0.205339 |
>> | GET
>> http://example.com:9292/v2/images?marker=eb00ee00-4401-4e59-bf92-602be2e45c4a
>> | 0.204854 |
>> | GET
>> http://example.com:9292/v2/images?marker=edf7d33a-286a-4a00-a8d8-4b0055d06093
>> | 0.203524 |
>> | GET
>> http://example.com:9292/v2/images?marker=75ab289d-90ce-496c-97a4-6fdd247c862a
>> | 0.215708 |
>> | GET
>> http://example.com:9292/v2/images?marker=b4eaf24c-94dc-4f2f-bd54-3eecbf290d75
>> | 0.28073 |
>> | GET http://example.com:8774/v2.1/flavors/detail?is_public=None
>> | 0.641211 |
>> | Total
>> | 18.573306
>>
>> as you see nova api takes 16 seconds to load . I have one project with
>> 1100 instance on it
>>
More information about the openstack-discuss
mailing list