Open Stack

Thanks Eric: +1, and I appreciate the peace-brokering :-)

I'm not wedded to the idea of putting "service metadata" into the same
collection as "user metadata"; it just seems like a reasonable approach to
me.  But it's more important to me that we agree on something, than that we
agree on the "best" thing!

My votes:

> Do we need to support variable service metadata?

+1  We saw a few people pop up on the list with their own use cases (e.g.
GPUs) when I proposed this originally.

> If so, where should it go?

+1 for re-using the same "Metadata" collection, with an "openstack:" prefix.
 Like it or not, we really have no choice but to reserve the "aws:" prefix,
and to be compatible with anything AWS decides to use this for in future.

> If in a key/value list, should existing instance properties be moved into
this format as well?

-1 (for now): We'd still have to support the separate properties for the
front end APIs.  We'd probably not want to refactor the database schema.
 However, the code can internally harmonize all the properties into a single
collection, which might make routing & processing requests easier.  We could
also support passing e.g. the instance types through the key/value
collection instead of the 'legacy' explicit parameter method in the
OpenStack API. However, I don't think we should change anything here yet,
until we have a clear rationale for doing so.  (Though this rationale might
arise quickly, e.g. multi-cluster-in-a-region)

> If in a key/value list, should it just be a prefix in a general metadata
list (and user metadata has a 'user' prefix)?

+1, but I don't think that we should require user metadata to be prefixed
(at least externally!)  We should reserve the "aws:" and "openstack:"
prefixes asap.  I'm going to propose a doc change to reserve the "aws:"
prefix - I think we have no choice in the matter, and I haven't seen any
objections.


Justin






On Wed, Mar 2, 2011 at 9:43 AM, Eric Day <eday at oddments.org> wrote:

> Well said Jorge. I think we're all in agreement that we need a way
> to add both user-defined metadata and service-specific metadata (and
> possibly deployment-specific metadata). I think Justin was working
> on the metadata mechanisms assuming it would support both based on
> prefix. If we don't want to overload metadata and use prefixes to
> differentiate between users, providers, and so forth, we should add
> another collection to resources. For example:
>
> metadata = [
>  "user:comment": "This is Eric's dev server"
>  "openstack:affinity_id": "rack4.dc2.east.rackspace.com"]
>
> Would become:
>
> user_metadata = ["comment": "This is Eric's dev server"]
> service_metadata = ["openstack:affinity_id": "rack4.dc2.east.rackspace.com
> "]
>
> Or don't use metadata for service metadata at all and put directly
> in the instance object/record:
>
> instance.id = 100
> instance.network = 42
> instance.affinity_id = rack4.dc2.east.rackspace.com
> # instance.metadata is user-defined metadata only.
> instance.metadata = ["comment": "This is Eric's dev server"]
>
> Now the arguments stated by many folks is that "service_metadata"
> is really instance properties or instance attributes and should
> instead be part of the instance object/record directly (like size,
> flavor id, etc. are). I don't disagree, but unfortunately there is
> a little more overhead since we're using a structured data store,
> and this requires an alter table for every change at this point. It's
> more difficult to introduce, test, and remove service attributes. If
> we want deployments to be able to define service-specific metadata
> and use that in pluggable schedulers, a DB schema change is not a
> very elegant way to support this.
>
> So the questions are:
>
> Do we need to support variable service metadata? If so, where should
> it go?
>
> If in a key/value list, should existing instance properties be moved
> into this format as well?
>
> If in a key/value list, should it just be a prefix in a general
> metadata list (and user metadata has a 'user' prefix)?
>
> -Eric
>
> On Wed, Mar 02, 2011 at 08:59:09AM -0500, Mark Washenberger wrote:
> > > [W]e
> > > shouldn't be overloading that functionality by performing some action
> based on
> > > user-defined metadata.
> >
> > That is exactly what I've been trying to say, but you have stated it much
> more succinctly. Thanks!
> >
> > My specific concern is with quotas. If the current osapi metadata is
> overloaded with api functionality, then it muddies the concept of quotas.
> Admins who run nova probably don't want the server affinity feature to count
> against the general metadata quota.
> >
> > Because of this, I don't think server affinity should be part of the
> metadata (specifically defined as the metadata attribute in
> nova.compute.api.API::create).
> >
> > This doesn't really have any effect on which of Jorge's options we
> choose, though, so /unhijack.
> >
> > "Jorge Williams" <jorge.williams at rackspace.com> said:
> >
> > > Metadata in the OpenStack Compute/Cloud Servers  API is meant to
> describe user
> > > defined properties.  That's it.  So in that case, I agree with Brian
> that we
> > > shouldn't be overloading that functionality by performing some action
> based on
> > > user-defined metadata.
> > >
> > > Speaking more generally though, any attribute that you associate with a
> resource
> > > can be thought of as metadata as well.  Isn't the name of an instance
> metadata
> > > about the instance?  Should operators be able to define arbitrary
> metadata and
> > > then be able to act on it in some way?  I think so, that's a very
> powerful
> > > feature. That said,  I would be cautious about exposing this as an
> arbitrary set
> > > of name value pairs because it provides a means by which you can bypass
> the
> > > contract and that will cause grief for our clients.  Additionally,
> there's the
> > > possibility of clashing metadata names between deployments.  The idea
> behind
> > > extensions is that you can define arbitrary metadata about a resource,
> while
> > > maintaining a contract and while avoiding conflicts with other
> > > operators/deployments/implementations.  I should note that the approach
> really
> > > isn't that different from AWS in that essentially as an operator you
> use a prefix
> > > to separate your metadata from customer metadata...the prefix is simply
> defined by
> > > the extension and  you can present your metadata in a separate
> attribute or
> > > element in the message.
> > >
> > > Given that, I'm still a little fuzzy about whether we've reached a
> decision as to
> > > whether affinity id:
> > >
> > > 1) Should be part of the core Compute API
> > > 2) Should be a more general concept that may span different services,
> as Eric Day
> > > proposes
> > > 3) Should be introduced as an extension, which can later be promoted to
> the
> > > core...or not :-)
> > >
> >
> >
> >
> > _______________________________________________
> > Mailing list: https://launchpad.net/~openstack
> > Post to     : openstack at lists.launchpad.net
> > Unsubscribe : https://launchpad.net/~openstack
> > More help   : https://help.launchpad.net/ListHelp
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack at lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20110302/b3de1656/attachment.html>