Open Stack

On Sat, 21 Jul 2012, Matt Joyce wrote:

> Preamble:
>
> Until now, all data that is made available by the metadata server has been
> data that cannot be found anywhere else at the time it may be needed.
>
> In short, an instance can't be passed it's instance id before it's instance
> id has been allocated so a user cannot pass it to an instance that is being
> started up.  So whether a user wants to jump through a few hoops or not to
> pass their instance the instance id of itself... they simply cannot without
> metadata api being there to provide it at creation time.
>
> This means that the metadata server holds an uneasy place as a necessary
> clearing house ( evil? ) of data that just doesn't have another place to
> be.  It's not secure, it's not authenticated, and it's a little scary that
> it exists at all.
>
> I wish to add some data to the metadata server that can be found somewhere
> else.  That a user could jump through a hoop or two to add to their
> instances.  Esteemed personages are concerned that I would be crossing the

Thank you for using 'Esteemed' rather than 'no good @#@$! %!$@$ %#%#%#%',
which I'm sure you considered.

> rubicon in terms of opening up the metadata api for wanton abuse.  They are
> not without a right or reason to be concerned.  And that is why I am going
> to attempt to explicitly classify a new category of data that we might wish
> to allow into the metadata server.  If we can be clear about what we are
> allowing we can avoid abuse.

As Jay was confused, I think we really need to separate the notion of "EC2
API" and "EC2 Instance Metadata API".  Right now, nova is *very* usable
without use of the external EC2 API.  It is much more dependent on the
Instance Metadata API.

Config drive is one alternative to the EC2 metadata service.

Matt's mail discusses only the "Instance Metadata API"

I had a blueprint at the last summit that covered making config-drive and
the ec2 metadata service basically contain the same data.
  http://etherpad.openstack.org/FolsomNovaConfigDriveImprovements

The general idea there was to replace the current config-drive format with
basically a rendering of the data found in the metadata service.  This
proposal met with surprisingly little complaint.  I planned on then also
extending that by adding to both the metadata service and the config-drive
"openstack metadata content".

This bit of information that Matt is interested in would fit under the
openstack metadata content.  But would not be available under the EC2
data.

My plan for backwards compatibility was basically that we would make:
  * http://169.254.169.254/ : look just like the ec2 metadata like thing
    that we have now in nova.  A request here would dump the same content
    it currently does, which is a list of YYYY-MM-DD versions.
  * http://169.254.169.254/ec2 point to http://169.254.169.254/
  * http://169.254.169.254/openstack/ be the well known location of the
    openstack metadata service.

The config drive basically then looks just like the metadata service.

We then have to define the content that we have at that exists at
/openstack, basically creating the first version of the openstack metadata
service api.  I personally see nothing wrong with YYYY-MM-DD for a
version, as it sorts so well, and is generally immediately understood what
it means.

I planned to simply have the top level entry list versions available
just like the EC2 one does.  Ie, It would have something like:
  2012-07-25
  latest
A request to http://169.254.169.254/openstack/2012-07-25 would return
JSON formatted data.  That data would have things like:
      instance-id
      ami-id
      keystone-endpoint <-- for Matt

What I said to Matt was that I think we need to consider (and scrutinize)
changes to the data available in the openstack metadata service the same
way we would an API change.

My choice of the keyname 'keystone-endpoint' is probably bad, but in
making a bad choice, it shows the value in making good choices, which
aren't made without thought and deliberation.

I simply haven't gotten around to doing the work above.  If someone else
wants to take a stab at it, I'd love to help and work together on it.

> My current work effort in regards to this is related to passing keystone
> credentials to instances via pam authentication.  So I can do a number of
> API related queries into openstack because I have credentials available to
> the OS that are dynamically allocated.  But to make my image portable I
> need to not be baking in the keystone API URI.

I promised Matt I would not complain, so my comments here have been deleted. :)