Open Stack

> Agreed. I wish people didn't use the EC2 API at all, since it's a
> complete bag of fail and a beautiful example of a terribly thought-out
> API. That said, the OpenStack Compute API v2 has its share of pockmarks
> to be sure.
>
> But... unfortunately, if you're going to use the EC2 API this hard-coded
> 169.254.169.254 address is what we have to deal with.
>
>
Agreed on all counts.


> > I wish to add some data to the metadata server that can be found
> > somewhere else.  That a user could jump through a hoop or two to add to
> > their instances.  Esteemed personages are concerned that I would be
> > crossing the rubicon in terms of opening up the metadata api for wanton
> > abuse.  They are not without a right or reason to be concerned.  And
> > that is why I am going to attempt to explicitly classify a new category
> > of data that we might wish to allow into the metadata server.  If we can
> > be clear about what we are allowing we can avoid abuse.
> >
> > I want to provide a uniform ( standardized? ) way for instances in the
> > openstack cloud to communicate back to the OpenStack APIs without having
> > to be provided data by the users of the cloud services.
>
> Let's be clear here... are you talking about the OpenStack Compute API
> or are you talking about the OpenStack Metadata service which is merely
> the EC2 Metadata API? We already have the config-drive extension [1]
> that allows information and files to be injected into the instance and
> loaded as a readonly device. The information in the config-drive can
> include things like the Keystone URI for the cell/AZ in which an
> instance resides.
>
>
I mean the OpenStack Metadata service.  The config drive extension does not
as far as I am aware produce a "uniform" path for data like this.  This API
query should be the same from openstack deployment to openstack deployment
to ensure portability of instances relying on this API query to figure out
where the catalog service is.  By "uniform" I mean it has all the love care
and backwards versioning support as a traditional API query.  The
config-drive seems more intended to be user customized rather than
considered a community supported API query.


> > Today the
> > mechanism by which this is done is catastrophically difficult for a new
> > user.
>
> Are you specifically referring here to the calls that, say, cloud-init
> makes to the (assumed to be running) EC2 metadata API service at
> http://169.254.169.254/latest/? Or something different? Just want to
> make sure I'm understanding what you are referring to as difficult.
>
>
I am referring to the whole new user experience.  Anything custom to a
deployment of openstack is now outside of our control and is not portable.
Also a new user will not be prepared to inject user data properly.  Going
further and a bit onto an irate tangent.  Horizon has a really round about
and completely non intuitive way of providing users with info on where API
servers are.  IE you have to generate an openstack credentials file.
download it.  and look at it in a text editor and then know what it is you
are looking at.  To find your tenant_name you have to guess in the dark
that horizon is referring to your tenant name as a project.  The whole
thing is insane.  What I am talking about here is a first step in allowing
image builders to integrate into openstack in a uniform way across all
installations ( or most ).  And that will allow people to reduce the
overall pain on new users of cloud at their pleasure.  I am asking for this
based on my experience trying to do this outside of openstack development.


> > This uniform way for instances to interact with the openstack API that I
> > want already sort of exists in the keystone catalog service.  The
> > problem is that you need to know where the keystone server is in the
> > world to access it.  That of course changes from deployment to
> > deployment.  Especially with the way SSL endpoints are being handled.
>
> This can be done using config-drive and the OpenStack community coming
> up with a standard file or tool that would be injected into the config
> drive. This would be similar to the calls currently executed by
> cloud-init that are hard-coded to look for 169.254.169.254. Would that
> work?
>

I don't know.  I'd say maybe.  But I'd prefer it was tracked as an API
call.  It will have in that area legitimate support from the community and
backwards version compatibility requirements.  I think ultimately it
belongs in the API as much as any other query.  While I think this sort of
solves the issue and may smooth a few folks feathers about the issue.  I
think it's probably the wrong way to handle it and likely to bite us in the
ass down the road when someone starts mangling that file or doesn't realize
config-drive is a dependency for that sort of fundamental query.


> > But the metadata API server is generally known as it uses a default ip
> > address value that can be found on any amazon compatible deployment.  In
> > fact to my knowledge it is the only known way to query openstack for
> > data relevant to interacting with it without user interaction.  And
> > that's the key to this whole thing.  I want to direct users or
> > automation baked into instances to the keystone api and catalog
> > service.  And the only way I know how to do that is the metadata service.
>
> As mentioned above, config-drive extension was built for just this
> purpose IIRC. Chris Macgown, who wrote the original extension, cc'd,
> should be able to comment on this further.
>
>
I disagree on that being the purpose.  I think the config drive is a user
customization option.  In this case I am talking openstack community
support like any other API query we support.  As far as I can tell placing
it as part of config-drive robs it of that.  And that's unacceptable for
people wanting to use that query for code baked into images.


> > This api data can be classified as being first and foremost OpenStack
> > infrastructure related.  Additionally it is not available without a user
> > providing it anywhere else.  And finally it is a catalog service.
> >
> > I'd love some more input on whether this makes sense, or can be improved
> > upon as an idea and formalized as a rule for using the metadata api
> > without abusing it.
>
> Well, we know we can't change the EC2 Metadata API since we don't own or
> have any control over the Amazon APIs. We can however come up with an
> OpenStack-centric tool using config-drive and a tool that would query a
> Keystone endpoint for a local OpenStack Compute API endpoint and then
> use the existing OpenStack Compute API calls for server metadata [2]?
>
> That sounds doable to you?
>
>
Not as it currently exists.  Unless config-drive is going to be held to the
same standard as an API query set.  And currently it may be in terms of
actually development support... but there is no authoritative community
promise to that.

I get the argument for using config-drive.  And to be blunt I think it's
probably superior to the metadata API on a number of things.  But it's only
functional in some reduced set of openstack deployments, it's not held to
the same level of scrutiny as an API query set.  And that means it's not
going to be usable today.  It  means it MIGHT be usable sometime in the
future maybe but as of now probably not.


> Best,
> -jay
>
> [1] Config Drive extension:
> http://docs.openstack.org/developer/nova/api_ext/ext_config_drive.html
> [2] Server metadata calls in Compute API:
>
> http://docs.openstack.org/api/openstack-compute/2/content/List_Metadata-d1e5089.html
>
>
> > Cheers,
> >
> >    Matt Joyce
> >
> > PS:
> >
> > My current work effort in regards to this is related to passing keystone
> > credentials to instances via pam authentication.  So I can do a number
> > of API related queries into openstack because I have credentials
> > available to the OS that are dynamically allocated.  But to make my
> > image portable I need to not be baking in the keystone API URI.
> >
> > If that gives any insight on why this is important to me.  Or possibly
> you.
> >
> >
> >
> >
> > _______________________________________________
> > Mailing list: https://launchpad.net/~openstack
> > Post to     : openstack at lists.launchpad.net
> > Unsubscribe : https://launchpad.net/~openstack
> > More help   : https://help.launchpad.net/ListHelp
> >
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack at lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20120723/12cea55b/attachment.html>