<font size=2 face="sans-serif">Hi Lincoln,</font>
<br>
<br><font size=2 face="sans-serif">Glad to see this thread rebooted. Here
are some comments on the metadata search API proposal. I would be glad
to see feedback from others too.</font>
<br>
<br><font size=2 face="sans-serif"><b>Data Types</b></font>
<br><font size=2 face="sans-serif">The proposal suggests each attribute
has one of 3 types: string, numeric, date. How do we deduce the type for
user metadata attributes i.e. object_meta_<name> attributes etc.?
This is important for range searches since types such as dates are not
sorted correctly when represented as strings. This seems to warrant a configuration
file which contains schema information. In addition, using attribute naming
conventions could help set the default behavior e.g. object_meta_start_date
could refer to a date by default since it has the suffix _date. Also, additional
types which seem important are a floating point type and a type for representing
geo-spatial coordinates and/or general spatial information. Ideally the
set of types which can be handled should be extensible. </font>
<br>
<br><font size=2 face="sans-serif"><b>System Metadata</b></font>
<br><font size=2 face="sans-serif">Regarding system metadata, many attributes
are part of this proposal which are not currently part of Swift. There
should be a minimal set of attributes - probably those attributes which
are part of Swift - which should be available for indexing by all vendor
implementations. Other than that, different vendors would have different
requirements for generating new system metadata for indexing, and it probably
shouldn't be a mandatory part of the API. Also, in general, the user should
be able to control which user and system metadata attributes are indexed
in order to control the amount of resources invested in metadata indexing/search.</font>
<br>
<br><font size=2 face="sans-serif">Let me know what you think.</font>
<br>
<br><font size=2 face="sans-serif">Best regards</font>
<br><font size=2 face="sans-serif">Paula</font>
<br>
<br><font size=2 color=#0080ff face="sans-serif"><b>Paula Ta-Shma, Ph.D.</b></font>
<br><font size=2 color=#0080ff face="sans-serif">Storage Research</font>
<br><font size=2 color=#4181c0 face="sans-serif">IBM Research - Haifa</font>
<br><font size=2 color=#4181c0 face="sans-serif">Phone:</font><font size=2 color=#5f5f5f face="sans-serif">
+972.3.7689402</font>
<br><font size=2 color=#4181c0 face="sans-serif">Email</font><font size=2>:
</font><font size=2 color=#5f5f5f face="sans-serif">paula@il.ibm.com</font>
<br>
<br>
<br>
<br>
<br><font size=1 color=#5f5f5f face="sans-serif">From:
</font><font size=1 face="sans-serif">"Thomas, Lincoln
(HP Storage R&D)" <Lincoln.Thomas@hp.com></font>
<br><font size=1 color=#5f5f5f face="sans-serif">To:
</font><font size=1 face="sans-serif">"openstack-dev@lists.openstack.org"
<openstack-dev@lists.openstack.org>, </font>
<br><font size=1 color=#5f5f5f face="sans-serif">Date:
</font><font size=1 face="sans-serif">11/01/2014 04:09 AM</font>
<br><font size=1 color=#5f5f5f face="sans-serif">Subject:
</font><font size=1 face="sans-serif">[openstack-dev]
[Swift] Metadata Search API</font>
<br>
<hr noshade>
<br>
<br>
<br><tt><font size=2>Rebooting this thread now that I've reorg'd the Wiki
page.<br>
<br>
The proposed REST API spec for searching system and custom metadata in
Swift, across accounts, containers, and objects, is now posted at:<br>
<br>
</font></tt><a href=https://wiki.openstack.org/wiki/MetadataSearchAPI><tt><font size=2>https://wiki.openstack.org/wiki/MetadataSearchAPI</font></tt></a><tt><font size=2><br>
<br>
I've also made the first modification to the API since the Icehouse design
summit where I introduced this project: adding "metadata:" per
conversation with Paula Ta-Shma's team at IBM Storage Research.<br>
<br>
The home page for this project remains at:<br>
<br>
</font></tt><a href=https://wiki.openstack.org/wiki/MetadataSearch><tt><font size=2>https://wiki.openstack.org/wiki/MetadataSearch</font></tt></a><tt><font size=2>
<br>
<br>
See that home page for further details, and the history of this email thread.
Feel free to edit the Wiki as described on the home page!<br>
<br>
As Brian Cline (SoftLayer) said so eloquently in this thread:<br>
<br>
> Today, about the best one can do is iterate through everything and
inspect metadata <br>
> along the way - obviously an infinitely expensive (and hilariously
insane) <br>
> operation.<br>
<br>
> If there are any others who have implemented search in Swift, please
speak up and <br>
> help shape this. We both want to get community consensus on a standard
search API, <br>
> then get a pluggable reference implementation into Swift.<br>
<br>
> This is all work-in-progress stuff, but we'd welcome any feedback,
concerns, <br>
> literal jumps for joy, etc. in this thread, both on the API and on
a reference <br>
> architecture.<br>
<br>
Thanks in advance,<br>
Lincoln Thomas (IRC lincolnt)<br>
System/Software Engineer, HP Storage R&D<br>
Portland, OR, USA, +1 (503) 757-6274<br>
<br>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
OpenStack-dev@lists.openstack.org<br>
</font></tt><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev"><tt><font size=2>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</font></tt></a><tt><font size=2><br>
<br>
</font></tt>
<br>