Open Stack

Swift uses marker+limit for pagination when listing containers or objects (with additional support for prefix, delimiters, and end markers). This is done because the total size of the listing may be rather large, and going to a correct "page" based on an offset gets expensive and doesn't allow for repeatable queries.

Pagination implies some sort of ordering, and I'm guessing (assuming+hoping) that your listings are based around something more meaningful that an incrementing id. By itself, "metric number 32592" doesn't mean anything, and listings like "go to metric 42000000 and give me the next 768 items" doesn't tell the consumer anything and probably isn't even a very repeatable query. Therefore, using a marker+prefix+limit style pagination system is very useful (eg "give me up to 1000 metrics that start with 'nova/instance_id/42/'"). Also, end_marker queries are very nice (half-closed ranges).

One thing I would suggest (and I hope we change in Swift whenever we update the API version) is that you don't promise to return the full page in a response. Instead, you should return a "no matches" or "end of listing" token. This allows you the flexibility to return responses quickly without consuming too many resources on the server side. Clients can then continue to iterate over subsequent pages as they are needed.

Something else that I'd like to see in Swift (it was almost added once) is the ability to reverse the order of the listings so you can iterate backwards over pages.

--John




On Nov 13, 2013, at 2:58 AM, Julien Danjou <julien at danjou.info> wrote:

> Hi,
> 
> We've been discussing and working for a while on support for pagination
> on our API v2 in Ceilometer. There's a large amount that already been
> done, but that is now stalled because we are not sure about the
> consensus.
> 
> There's mainly two approaches around pagination as far as I know, one
> being using limit/offset and the other one being marker based. As of
> today, we have no clue of which one we should pick, in the case we would
> have a technical choice doable between these two.
> 
> I've added the Horizon tag in the subject because I think it may concern
> Horizon, since it shall be someday in the future one of the main
> consumer of the Ceilometer API.
> 
> I'd be also happy to learn what other projects do in this regard, and
> what has been said and discussed during the summit.
> 
> To a certain extend, we Ceilometer would also be happy to find common
> technical ground on this to some extend so _maybe_ we can generalise
> this into WSME itself for consumption from other projects.
> 
> Cheers,
> -- 
> Julien Danjou
> ;; Free Software hacker ; independent consultant
> ;; http://julien.danjou.info
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131113/79c1c6e0/attachment.pgp>