Open Stack

On Fri, 17 Jun 2011, Thorsten von Eicken wrote:

> We're very much looking forward to the new "portable configuration
> drive" functionality and would like to provide feedback. If this is not
> the best forum, please point me to it.
>
> The blueprint
> is: https://blueprints.launchpad.net/nova/+spec/configuration-drive
> We reviewed the initial work
> in: https://github.com/ChristopherMacGown/nova/commit/47c041a642ff32085b3140475d7a2a62dcb62c1a
>
> Feedback:
>

This spec is very similar to implementing what is "OVF Transport".  So
similar in fact that I think it should/could be made compatible.
Read more at
http://www.dmtf.org/standards/published_documents/DSP0243_1.1.0.pdf .

> 1. It is not always obvious how to enumerate attached devices and the
> document doesn't explicitly cover how we'd determine which attached
> device represents the configuration volume. Specifically consider
> Windows as well.

I am not very knowledgeable about windows.  However, I'd think as long as
the volume doesn't end up being the "boot volume" ("C:") or whatever, then
you do not need to worry.  The data on the disk should be formatted in
such a way that it is identifiable.  That can be done in a number of ways:
 * filesystem label (iso9660 and even i think FAT have a label)
 * contents of the filesystem

The guest OS can be expected to go searching around for its configuration
data.

> 2. Will the configuration drive be available at boot time on a new
> instance? Or does it appear "later" (that would be bad)?

It needs to be available immediately on first boot. It is very difficult
to figure out that the guest is done with the volume, so it should remain
attached until the user did some explicit action to detach it.  I really
like the suggestion in the etherpad that it is a "normal volume", and then
it can be detached via api call.

> 3. How does one send the configuration drive content?  What is the API
> call where we provide the configuration information and what is the
> expected format?

In another message on this thread, Christopher said :
>> a small ~64MB ISO formatted volume that would be attached and mounted
>> read-only by the guest instance on boot

I think we can greatly increase the functionality of this feature by
removing restrictions.   The essence of this feature is that you're
allowing the user to provide you with some data that you'll then put on a
volume, attach the volume to the guest, and boot the guest.

That can all be done today within AWS with a utility instance, by doing:
 * launch utility instance
 * attach volume to utility instance
 * populate volume via 'mkisofs' or anything else inside the instance
 * snapshot volume
 * [ delete original volume ]
 * launch instance with --block-device-mapping="....snap-abcdefg"

So, to me, the only value of this spec is being able to do that list of
things above without the utility instance.

The big AHA should come now.  This is *very generic function*.  I would
suggest 3 entry points to the API that I think cover the vast majority of
cases here without limiting you.  Forgive the very bad

A.) volumeID PopulateVolume(volumeSize, volumeContent)
   This basically takes content which will then be seen from the volume
   when read from the guest.  If the user did
      'dd bs=1 count=$volumeSize if=$devName',
   with correct values for devName and volumeSize, they would get exactly
   "volumeContent".
   It is quite reasonable to limit this to some small size to avoid people
   populating massive disks with it.  Some other entry point to this same
   function might be to allow the user to specify a S3 file that they wish
   to populate a volume from.  That way, you have whatever more robust
   mechanisms are available to populate S3 with, and the API call is
   small.
B.) instanceId RunInstances(....., createVolumeSize, createVolumeContent)
   This is basically convenience wrapper around the PopulateVolume above.
   That just creates the volume, attaches it to this instance id.  Amazon
   has ways of specifying what should happen to this volume when the
   instance dies (so you can specify that it will disappear with the
   instance and dont have to worry about cleaning it up)

   It basically does 'PopulateVolume(createVolumeSize,createVolumeContent)'
   and then attaches the volume.
C.) instanceId RunInstances(....., fileContent, fileName="config.json",
       fsType="iso9660")
   Another convenience wrapper around PopulateVolume above.  This does
   the same as the RunInstances above, but created the filesystem for the
   user.  That way the user doesn't have to worry about creating a
   ISO9660 or vfat filesystem.  2 filesytems that would be expected to be
   supported would be vfat and iso9660.
D.) instanceId RunInstances(.... configData)
    This is just a wrapper around 'C', that takes only ConfigData and does
    the rest.

I feel fairly strongly that there is no reason to artificially limit the
data to a given filesystem type, filename, or file format.

If I could specify either:
 * file content like seen at :
    http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/doc/ovf/environment.xml
 * filename "ovf-env.xml"
 * filsytem format IS9660
or
 * file blob (containing ISO9660 data that I craft) with the same as above
Then I can use cloud-init's "OVF Transport" support, It wouldn't surprise
me of other code that uses OVF Transport would work given that same
function.

>
> 4. It looks like the configuration content is placed into a config.json
> file, is this correct? Is that a blob passed through the API? Does it
> have to be json or is that just a convention? What is the max size?

Please, a blob. Build in as few expectations on that data as possible.

>
> 5. What are the permissions required to access the configuration drive?
> Would be ideal for the content to be root/administrator accessible only
> (because it may contain credentials). Can this be influenced via the API?

No it cannot.  This is entirely up to the guest, and it *should* be.  It
is assumed that the user has elevated privileges if they can read disk
volumes outside of the filesystem.

> 6. Is this device going to be read-only?

I'm neither here nor there on that.  I dont see any good reason to limit
it.
>
> 7. We need to be able to change the configuration content when
> stopping&starting instances, will this be possible?

If you use "normal volumes" for this, then with function in AWS, you could
just stop the instance, attach it to another instance, modify, start....

More generically, if these are generic volumes, you could provide a
'volumeId' input to 'PopulateVolume' (or any of the other suggested
functions above , and if the volume was not attached the hypervisor would
make that volume have the specified data.

> 8. There is a related spec: 'instance-transport'. Is that still
> relevant? Or superseded by the configuration drive?

It is relevant. I think sometime in the future, there will be a need for a
good 2 way communication mechanism.

> 9. Will the configuration drive contain any metadata? E.g. IP addresses,
> names of available volumes, instance ID, etc.

This is one thing that 'PopulateVolume' up above can't handle easily, as
the user doesn't know the instance id or volume id when they make that
request.  That would mean either such run-time data would need to come
from another more dynamic source (the existing metadata service)/.

One other thing I'd like to mention, I personally feel that, by default,
these drives with "config data" should be "ephemeral" by default.  The
volume should die with the instance unless other action is taken by the
user.