I am experiencing the same behavior on all of my clouds. It's normal I believe because behind the command so many api calls go out to construct data. We have 500 vms in single projects so I wrote my own wrapper script to talk to mariadb and fetch that information directly from DB. This is not the best solution but works for me and now i can pull out any records within microseconds.
$ mysql -h 172.28.15.79 -u user1 --password=<PASSWORD> nova -e "select display_name as hostname,vm_state,node,availability_zone,vcpus,memory_mb,root_gb from instances WHERE (vm_state='active' OR vm_state='stopped');"