BTW, If the source image is based on compression or thin provision type (like VDI, QCOW2, VMDK, etc.) It will take a long time to create no matter boot on image or volume. Nova will convert the image based on these type first during creation. Because Ceph RBD doesn't support. Make sure all the images you upload is based on RBD format (or RAW format in other word), unless the virtual size of image is small. . Tony Liu <tonyliu0592@hotmail.com> 於 2021年4月1日 週四 上午10:18寫道：

Thank you Eddie! It makes sense. Creating a snapshot is much faster than copying image to a volume.

Tony ________________________________________ From: Eddie Yen <missile0407@gmail.com> Sent: March 31, 2021 05:59 PM To: Tony Liu Cc: openstack-discuss@lists.openstack.org Subject: Re: launch VM on volume vs. image

Hi Tony,

In Ceph layer, IME, launching VM on image is creating a snapshot from source image in Nova ephemeral pool. If you check the RBD image created in Nova ephemeral pool, all images have their own parents from glance images.

For launching VM on volume, it will "copy" the image to volume pool first, resize to specified disk size, then connect and boot. Because it's not create a snapshot from image, so it will take much longer.

Eddie.

Tony Liu <tonyliu0592@hotmail.com<mailto:tonyliu0592@hotmail.com>> 於 2021年4月1日 週四 上午8:09寫道： Hi,

With Ceph as the backend storage, launching a VM on volume takes much longer than launching on image. Why is that? Could anyone elaborate the high level workflow for those two cases?

Thanks! Tony

I uploaded 800GB RAW image. Then launching VM on either image or volume is in 10 seconds. Tony ________________________________________ From: Tony Liu <tonyliu0592@hotmail.com> Sent: April 1, 2021 05:24 PM To: Eddie Yen Cc: openstack-discuss@lists.openstack.org Subject: Re: launch VM on volume vs. image I have a 300GB QCOW image (800GB raw space). If launch VM on volume, Cinder will need to convert it first, and that requires at least 300GB free disk space on controller. If launch VM on image, it takes forever, I didn't look into where it's stuck. Is there any easier way to launch VM from such image? Ceph is the storage backend. Thanks! Tony ________________________________________ From: Eddie Yen <missile0407@gmail.com> Sent: March 31, 2021 10:47 PM To: Tony Liu Cc: openstack-discuss@lists.openstack.org Subject: Re: launch VM on volume vs. image BTW, If the source image is based on compression or thin provision type (like VDI, QCOW2, VMDK, etc.) It will take a long time to create no matter boot on image or volume. Nova will convert the image based on these type first during creation. Because Ceph RBD doesn't support. Make sure all the images you upload is based on RBD format (or RAW format in other word), unless the virtual size of image is small. . Tony Liu <tonyliu0592@hotmail.com<mailto:tonyliu0592@hotmail.com>> 於 2021年4月1日 週四 上午10:18寫道： Thank you Eddie! It makes sense. Creating a snapshot is much faster than copying image to a volume. Tony ________________________________________ From: Eddie Yen <missile0407@gmail.com<mailto:missile0407@gmail.com>> Sent: March 31, 2021 05:59 PM To: Tony Liu Cc: openstack-discuss@lists.openstack.org<mailto:openstack-discuss@lists.openstack.org> Subject: Re: launch VM on volume vs. image Hi Tony, In Ceph layer, IME, launching VM on image is creating a snapshot from source image in Nova ephemeral pool. If you check the RBD image created in Nova ephemeral pool, all images have their own parents from glance images. For launching VM on volume, it will "copy" the image to volume pool first, resize to specified disk size, then connect and boot. Because it's not create a snapshot from image, so it will take much longer. Eddie. Tony Liu <tonyliu0592@hotmail.com<mailto:tonyliu0592@hotmail.com><mailto:tonyliu0592@hotmail.com<mailto:tonyliu0592@hotmail.com>>> 於 2021年4月1日 週四 上午8:09寫道： Hi, With Ceph as the backend storage, launching a VM on volume takes much longer than launching on image. Why is that? Could anyone elaborate the high level workflow for those two cases? Thanks! Tony

Hi Dominic, What's your image format? Thanks! Tony ________________________________________ From: DHilsbos@performair.com <DHilsbos@performair.com> Sent: April 1, 2021 10:47 AM To: tonyliu0592@hotmail.com Cc: openstack-discuss@lists.openstack.org; missile0407@gmail.com Subject: RE: launch VM on volume vs. image Tony / Eddie; I think this is partially dependent on the version of OpenStack running. In our Victoria cloud, a volume created from an image is also done as a snapshot by Ceph, and is completed in seconds. Thank you, Dominic L. Hilsbos, MBA Director – Information Technology Perform Air International Inc. DHilsbos@PerformAir.com www.PerformAir.com From: Eddie Yen [mailto:missile0407@gmail.com] Sent: Wednesday, March 31, 2021 6:00 PM To: Tony Liu Cc: openstack-discuss@lists.openstack.org Subject: Re: launch VM on volume vs. image Hi Tony, In Ceph layer, IME, launching VM on image is creating a snapshot from source image in Nova ephemeral pool. If you check the RBD image created in Nova ephemeral pool, all images have their own parents from glance images. For launching VM on volume, it will "copy" the image to volume pool first, resize to specified disk size, then connect and boot. Because it's not create a snapshot from image, so it will take much longer. Eddie. Tony Liu <tonyliu0592@hotmail.com> 於 2021年4月1日 週四 上午8:09寫道： Hi, With Ceph as the backend storage, launching a VM on volume takes much longer than launching on image. Why is that? Could anyone elaborate the high level workflow for those two cases? Thanks! Tony

According to Tony's info, it will take a very long time to create because it needs to convert first. Like what Dominic said, not only wasting time but also wasting the compute node disk space in every creation. Still suggest converting to RAW format first, only a long time to upload the Ceph for the first time. <DHilsbos@performair.com> 於 2021年4月2日 週五 下午11:23寫道：

Tony;

In accordance with the second "Important" note listed here: https://docs.ceph.com/en/nautilus/rbd/rbd-openstack/, we use all RAW images.

Thank you,

Dominic L. Hilsbos, MBA Director ? Information Technology Perform Air International Inc. DHilsbos@PerformAir.com www.PerformAir.com

-----Original Message----- From: Tony Liu [mailto:tonyliu0592@hotmail.com] Sent: Thursday, April 1, 2021 5:19 PM To: Dominic Hilsbos Cc: openstack-discuss@lists.openstack.org; missile0407@gmail.com Subject: Re: launch VM on volume vs. image

Hi Dominic,

What's your image format?

Thanks! Tony ________________________________________ From: DHilsbos@performair.com <DHilsbos@performair.com> Sent: April 1, 2021 10:47 AM To: tonyliu0592@hotmail.com Cc: openstack-discuss@lists.openstack.org; missile0407@gmail.com Subject: RE: launch VM on volume vs. image

Tony / Eddie;

I think this is partially dependent on the version of OpenStack running. In our Victoria cloud, a volume created from an image is also done as a snapshot by Ceph, and is completed in seconds.

Thank you,

Dominic L. Hilsbos, MBA Director ? Information Technology Perform Air International Inc. DHilsbos@PerformAir.com www.PerformAir.com

From: Eddie Yen [mailto:missile0407@gmail.com] Sent: Wednesday, March 31, 2021 6:00 PM To: Tony Liu Cc: openstack-discuss@lists.openstack.org Subject: Re: launch VM on volume vs. image

Hi Tony,

In Ceph layer, IME, launching VM on image is creating a snapshot from source image in Nova ephemeral pool. If you check the RBD image created in Nova ephemeral pool, all images have their own parents from glance images.

For launching VM on volume, it will "copy" the image to volume pool first, resize to specified disk size, then connect and boot. Because it's not create a snapshot from image, so it will take much longer.

Eddie.

Tony Liu <tonyliu0592@hotmail.com> 於 2021年4月1日 週四 上午8:09寫道： Hi,

With Ceph as the backend storage, launching a VM on volume takes much longer than launching on image. Why is that? Could anyone elaborate the high level workflow for those two cases?

Thanks! Tony