Open Stack

> > You could be right there, but in any case good to air the
> > discussion, and hopefully scare up a couple more view-points.
> 
> 
> I'm going to summarize the discussion at the summit just so it is
> included here.
> 
> There are two things you really need to be able to specify for each
> of the drives in your instance:
> 
> 1) What is the source of the bits?
> 
>  * image in glance
>  * volume in cinder
>  * snapshot in cinder
>  * local file
>  * empty
>  * formatted ext4
> 
> 2) What is the destination of the volume (location and size):
>  
>  * permanent local disk
>  * ephemeral local disk
>  * remote volume
> 
> The confusion about boot-from-volume arises because the destination
> is interpreted
> from the source (and sometimes the source is interpreted from the
> instance type)
> 
> Examples:
>   If source is a snapshot in cinder, then the destination will be a
>   remote volume
>   If source is an image in glance, then the destination will be
>   permanent local disk
>   If it is the second drive of an instance, then the source will be
>   formatted ext4
> 
> We had a goal at the summit of making these things explicit,
> but this thread shows that there is one aspect we did not
> consider: where does the metadata about the boot source come
> from?
> 
> We store various metadata that we currently can store in
> glance, such as image format. The metadata is used in
> scheduling for example. It may also in the future be used to
> determine different emulation modes for hardware (as in images
> that don't support virtio networking).  The metadata needs to
> come from somewhere.
> 
> There are three options here:
> 
> 1) metadata is stored in glance
> 2) metadata is passed in at boot time
> 3) metadata is stored along with the bits somewhere (metadata in
> cinder)
> 
> It bears mentioning that we already support 1) you can create a
> metadata-only image today.
> We already have limited support for 2), you can set the bdm data at
> boot time.
> 
> I think overriding metadata at boot time is valuable, so i see no
> need to remove that ability.

Yep, I can see the desire not to remove an existing degree of
freedom.

Though IIUC the metadata-at-boot option is only built-out currently
in a limited way (i.e. only the BDM property), so I guess this would
need to be extended to include any other metadata that may be relevant?
 
> So the question boils down to whether we REQUIRE metadata in
> glance.  It definitely makes things simpler but it also makes
> the user experience quite painful.  If I want to boot with a
> source of a glance image and have a persistant volume. I have
> to go to cinder to create a volume from the image, then go to
> glance to create a new image backing to the volume, then go to
> nova to boot the new image. Yuck!

True enough, that does sound like a drag.

Though I guess we could orchestrate some of those operations into a 
single workflow step (e.g. create the volume, snapshot it, then create
the metadata-only image).
 
> It also adds extra round trips to glance that wouldn't be necessary
> if we omit the image_id.

True.

> To boot a vm, we need source bits. our image_id represents both
> the metadata and (sometimes) the source bits. If a user sends
> us sufficient metadata for us to find the source bits, I don't
> think we need to require putting metadata in glance. It seems
> like it might be a useful default however.

OK, that's fair enough. 

I think the metadata-only image is most useful when working with
a volume snapshot as opposed to a volume directly. In that case,
the purpose of the snapshot would generally be to allow multiple
instances to be booted off the same bootable volume, starting
from a pristine state each time. (Whereas IIUC booting direct from
a volume is more of a single-instance deal). The snapshot-backed
metadata-only image idea is also closer to the EC2 concept of a
EBS-backed AMI.

So we could potentially add a nova API extension that allows such a
"reusable" metadata-only image to be created direct from  a source
image ... i.e. create a volume from the source image, snapshot the
volume, create a metadata-only image referencing the snapshot.

Cheers,
Eoghan