[openstack-dev] [Swift] Metadata Search API
PAULA at il.ibm.com
Thu Jan 23 15:47:49 UTC 2014
Glad to see this thread rebooted. Here are some comments on the metadata
search API proposal. I would be glad to see feedback from others too.
The proposal suggests each attribute has one of 3 types: string, numeric,
date. How do we deduce the type for user metadata attributes i.e.
object_meta_<name> attributes etc.? This is important for range searches
since types such as dates are not sorted correctly when represented as
strings. This seems to warrant a configuration file which contains schema
information. In addition, using attribute naming conventions could help
set the default behavior e.g. object_meta_start_date could refer to a date
by default since it has the suffix _date. Also, additional types which
seem important are a floating point type and a type for representing
geo-spatial coordinates and/or general spatial information. Ideally the
set of types which can be handled should be extensible.
Regarding system metadata, many attributes are part of this proposal which
are not currently part of Swift. There should be a minimal set of
attributes - probably those attributes which are part of Swift - which
should be available for indexing by all vendor implementations. Other than
that, different vendors would have different requirements for generating
new system metadata for indexing, and it probably shouldn't be a mandatory
part of the API. Also, in general, the user should be able to control
which user and system metadata attributes are indexed in order to control
the amount of resources invested in metadata indexing/search.
Let me know what you think.
Paula Ta-Shma, Ph.D.
IBM Research - Haifa
Email: paula at il.ibm.com
From: "Thomas, Lincoln (HP Storage R&D)" <Lincoln.Thomas at hp.com>
To: "openstack-dev at lists.openstack.org"
<openstack-dev at lists.openstack.org>,
Date: 11/01/2014 04:09 AM
Subject: [openstack-dev] [Swift] Metadata Search API
Rebooting this thread now that I've reorg'd the Wiki page.
The proposed REST API spec for searching system and custom metadata in
Swift, across accounts, containers, and objects, is now posted at:
I've also made the first modification to the API since the Icehouse design
summit where I introduced this project: adding "metadata:" per
conversation with Paula Ta-Shma's team at IBM Storage Research.
The home page for this project remains at:
See that home page for further details, and the history of this email
thread. Feel free to edit the Wiki as described on the home page!
As Brian Cline (SoftLayer) said so eloquently in this thread:
> Today, about the best one can do is iterate through everything and
> along the way - obviously an infinitely expensive (and hilariously
> If there are any others who have implemented search in Swift, please
speak up and
> help shape this. We both want to get community consensus on a standard
> then get a pluggable reference implementation into Swift.
> This is all work-in-progress stuff, but we'd welcome any feedback,
> literal jumps for joy, etc. in this thread, both on the API and on a
Thanks in advance,
Lincoln Thomas (IRC lincolnt)
System/Software Engineer, HP Storage R&D
Portland, OR, USA, +1 (503) 757-6274
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the OpenStack-dev