Open Stack

Thanks Matt, comments inline...

On 07/23/2012 05:25 PM, Matt Joyce wrote:
>     > I wish to add some data to the metadata server that can be found
>     > somewhere else.  That a user could jump through a hoop or two to
>     add to
>     > their instances.  Esteemed personages are concerned that I would be
>     > crossing the rubicon in terms of opening up the metadata api for
>     wanton
>     > abuse.  They are not without a right or reason to be concerned.  And
>     > that is why I am going to attempt to explicitly classify a new
>     category
>     > of data that we might wish to allow into the metadata server.  If
>     we can
>     > be clear about what we are allowing we can avoid abuse.
>     >
>     > I want to provide a uniform ( standardized? ) way for instances in the
>     > openstack cloud to communicate back to the OpenStack APIs without
>     having
>     > to be provided data by the users of the cloud services.
> 
>     Let's be clear here... are you talking about the OpenStack Compute API
>     or are you talking about the OpenStack Metadata service which is merely
>     the EC2 Metadata API? We already have the config-drive extension [1]
>     that allows information and files to be injected into the instance and
>     loaded as a readonly device. The information in the config-drive can
>     include things like the Keystone URI for the cell/AZ in which an
>     instance resides.
> 
> I mean the OpenStack Metadata service.

Sorry, I'm still confused. The only actual stand-alone service in
OpenStack for metadata is the OpenStack EC2 Metadata service that runs
on the fixed AWS 169.254.169.254 address. Are you referring to this, or
are you referring to the /servers/<SERVER_ID>/metadata call in the
OpenStack Compute API v2?

> The config drive extension does
> not as far as I am aware produce a "uniform" path for data like this. 

Absolutely correct. The community would need to come to a consensus on
this uniformity, just as Amazon came to the decision to hard-code the
169.254.169.254 address.

> This API query should be the same from openstack deployment to openstack
> deployment to ensure portability of instances relying on this API query
> to figure out where the catalog service is.  By "uniform" I mean it has
> all the love care and backwards versioning support as a traditional API
> query.

Agree completely.

> The config-drive seems more intended to be user customized
> rather than considered a community supported API query.

Well, that may be the case, but as mentioned above, I think we could
*use* config-drive along with a community consensus on a uniform place
to store lookup information for a real OpenStack metadata service --
things like a private key, info file containing the IP of the "nearest"
metadata service, etc...

> 
>     > Today the
>     > mechanism by which this is done is catastrophically difficult for
>     a new
>     > user.
> 
>     Are you specifically referring here to the calls that, say, cloud-init
>     makes to the (assumed to be running) EC2 metadata API service at
>     http://169.254.169.254/latest/? Or something different? Just want to
>     make sure I'm understanding what you are referring to as difficult.
> 
> I am referring to the whole new user experience.  Anything custom to a
> deployment of openstack is now outside of our control and is not
> portable. 

Sure, completely agree!

> Also a new user will not be prepared to inject user data
> properly. 

Well, I'm actually not suggesting having the user really be involved at
all with the injection of keys/information into the config-drive :) That
would done by Nova.

When a user currently launches an image in OpenStack, that image
connects to the EC2 metadata service automatically if cloud-init is
installed in the image. I am picturing a similar scenario for this
config-drive stuff -- only instead of cloud-init needing to be installed
on the image, I'm suggesting Nova create a "standard" (uniform)
config-drive (or part of a config-drive) that contained upstart/startup
scripts, keys, and info for connecting to some OpenStack Metadata service.

> Going further and a bit onto an irate tangent.  Horizon has a
> really round about and completely non intuitive way of providing users
> with info on where API servers are.  IE you have to generate an
> openstack credentials file.  download it.  and look at it in a text
> editor and then know what it is you are looking at.  To find your
> tenant_name you have to guess in the dark that horizon is referring to
> your tenant name as a project. 

Heh, well, you know where I stand on the whole tenant vs. project theme
:) That said, it's a bit of a tangent, as you admit to above :)

>  The whole thing is insane.  What I am
> talking about here is a first step in allowing image builders to
> integrate into openstack in a uniform way across all installations ( or
> most ).  And that will allow people to reduce the overall pain on new
> users of cloud at their pleasure.  I am asking for this based on my
> experience trying to do this outside of openstack development. 

See my suggestion above... maybe we're getting closer towards a
commonly-envisioned solution? Snipping the rest of the conversation
because I basically agree with most of what you're saying.

Best,
-jay

>     > This uniform way for instances to interact with the openstack API
>     that I
>     > want already sort of exists in the keystone catalog service.  The
>     > problem is that you need to know where the keystone server is in the
>     > world to access it.  That of course changes from deployment to
>     > deployment.  Especially with the way SSL endpoints are being handled.
> 
>     This can be done using config-drive and the OpenStack community coming
>     up with a standard file or tool that would be injected into the config
>     drive. This would be similar to the calls currently executed by
>     cloud-init that are hard-coded to look for 169.254.169.254. Would
>     that work?
> 
> 
> I don't know.  I'd say maybe.  But I'd prefer it was tracked as an API
> call.  It will have in that area legitimate support from the community
> and backwards version compatibility requirements.  I think ultimately it
> belongs in the API as much as any other query.  While I think this sort
> of solves the issue and may smooth a few folks feathers about the
> issue.  I think it's probably the wrong way to handle it and likely to
> bite us in the ass down the road when someone starts mangling that file
> or doesn't realize config-drive is a dependency for that sort of
> fundamental query.
>  
> 
>     > But the metadata API server is generally known as it uses a default ip
>     > address value that can be found on any amazon compatible
>     deployment.  In
>     > fact to my knowledge it is the only known way to query openstack for
>     > data relevant to interacting with it without user interaction.  And
>     > that's the key to this whole thing.  I want to direct users or
>     > automation baked into instances to the keystone api and catalog
>     > service.  And the only way I know how to do that is the metadata
>     service.
> 
>     As mentioned above, config-drive extension was built for just this
>     purpose IIRC. Chris Macgown, who wrote the original extension, cc'd,
>     should be able to comment on this further.
> 
> 
> I disagree on that being the purpose.  I think the config drive is a
> user customization option.  In this case I am talking openstack
> community support like any other API query we support.  As far as I can
> tell placing it as part of config-drive robs it of that.  And that's
> unacceptable for people wanting to use that query for code baked into
> images.
>  
> 
>     > This api data can be classified as being first and foremost OpenStack
>     > infrastructure related.  Additionally it is not available without
>     a user
>     > providing it anywhere else.  And finally it is a catalog service.
>     >
>     > I'd love some more input on whether this makes sense, or can be
>     improved
>     > upon as an idea and formalized as a rule for using the metadata api
>     > without abusing it.
> 
>     Well, we know we can't change the EC2 Metadata API since we don't own or
>     have any control over the Amazon APIs. We can however come up with an
>     OpenStack-centric tool using config-drive and a tool that would query a
>     Keystone endpoint for a local OpenStack Compute API endpoint and then
>     use the existing OpenStack Compute API calls for server metadata [2]?
> 
>     That sounds doable to you?
> 
> 
> Not as it currently exists.  Unless config-drive is going to be held to
> the same standard as an API query set.  And currently it may be in terms
> of actually development support... but there is no authoritative
> community promise to that.
> 
> I get the argument for using config-drive.  And to be blunt I think it's
> probably superior to the metadata API on a number of things.  But it's
> only functional in some reduced set of openstack deployments, it's not
> held to the same level of scrutiny as an API query set.  And that means
> it's not going to be usable today.  It  means it MIGHT be usable
> sometime in the future maybe but as of now probably not.
>  
> 
>     Best,
>     -jay
> 
>     [1] Config Drive extension:
>     http://docs.openstack.org/developer/nova/api_ext/ext_config_drive.html
>     [2] Server metadata calls in Compute API:
>     http://docs.openstack.org/api/openstack-compute/2/content/List_Metadata-d1e5089.html
> 
> 
>     > Cheers,
>     >
>     >    Matt Joyce
>     >
>     > PS:
>     >
>     > My current work effort in regards to this is related to passing
>     keystone
>     > credentials to instances via pam authentication.  So I can do a number
>     > of API related queries into openstack because I have credentials
>     > available to the OS that are dynamically allocated.  But to make my
>     > image portable I need to not be baking in the keystone API URI.
>     >
>     > If that gives any insight on why this is important to me.  Or
>     possibly you.
>     >
>     >
>     >
>     >
>     > _______________________________________________
>     > Mailing list: https://launchpad.net/~openstack
>     <https://launchpad.net/%7Eopenstack>
>     > Post to     : openstack at lists.launchpad.net
>     <mailto:openstack at lists.launchpad.net>
>     > Unsubscribe : https://launchpad.net/~openstack
>     <https://launchpad.net/%7Eopenstack>
>     > More help   : https://help.launchpad.net/ListHelp
>     >
> 
>     _______________________________________________
>     Mailing list: https://launchpad.net/~openstack
>     <https://launchpad.net/%7Eopenstack>
>     Post to     : openstack at lists.launchpad.net
>     <mailto:openstack at lists.launchpad.net>
>     Unsubscribe : https://launchpad.net/~openstack
>     <https://launchpad.net/%7Eopenstack>
>     More help   : https://help.launchpad.net/ListHelp
> 
>