Open Stack

Ok, we have done a few things to make that process much faster.

Add in a lots of memory in Compute Node. That helps because the reading
speed of HDD is extremely slow. But if node has large DRMA, Linux is smart
to read large trunk of file into DRAM. 

In addition, we use network multicast, instead of N times p2p reading from
image resource, to distribute the image to ALL nodes simultaneously. So, the
provisioning time is not growing with number of node linearly anymore. 

We have a plan to use Software RDMA instead of TCP, later. 

With these approaches and 10gbe network, we see potential to boot 1000vms,
with 3G+ image size, in 60s

Code is not production read. We are trying them in larger cluster. 

BR
Leon 

-----Original Message-----
From: Turbo Fredriksson [mailto:turbo at bayour.com] 
Sent: Wednesday, September 28, 2016 3:58 PM
To: Leon Liu
Cc: openstack at lists.openstack.org
Subject: Re: [Openstack] Provisioning VMs in Openstack cluster

On Sep 28, 2016, at 8:34 PM, Leon Liu wrote:

> We are trying to provision a few VM images in a small Openstack 
> cluster. Our image is about 3GB size.  It has taken six minutes to 
> provision just three VMs, one on each host; and it grows linearly. 
> Now, we are wondering it may take 200 minutes for 100 VMs to be ready.
That would be slow.


I have the same experience, if that helps :(.

And my images is not any where near that size, only 463MB!

Using Heat, creating a volume for each instance on a remote SAN, 'takes
ages' (haven't clocked it, but creating eight instances takes well over half
an hour)!


I'm "fairly" certain that it's not the network - I have four trunked 1Gbps
links directly down to my storage (and I've triple checked that that is
used).


I've had other things on my mind and I'm not creating a lot of instances now
that most of them have been created, but I've always intended to go back to
this problem so I'm going to follow this thread closely.
--
Ehhhhm - The battle cry of the cronical masturbater.
- Charlie Harper