Open Stack

On Tue, 2014-03-18 at 13:30 -0500, Steven Kaufer wrote:
> Jay Pipes <jaypipes at gmail.com> wrote on 03/18/2014 12:02:50 PM:
> 
> > From: Jay Pipes <jaypipes at gmail.com>
> > To: openstack-dev at lists.openstack.org, 
> > Date: 03/18/2014 12:15 PM
> > Subject: Re: [openstack-dev] Offset support in REST API pagination
> > 
> > On Tue, 2014-03-18 at 11:31 -0500, Steven Kaufer wrote:
> > > First, here is some background on this topic:
> > >  http://www.gossamer-threads.com/lists/openstack/dev/2777
> > > 
> > > Does anyone have any insight as to why offset is not supported in
> the
> > > REST API calls that support pagination?   I realize that there are
> > > tradeoffs when using a offset (vs. marker) but I believe that
> there is
> > > value in supporting both.  For example, if you want to jump to the
> > > n-th page of data without having to traverse all of the previous
> > > pages.
> > > 
> > > Is there a reason why the APIs do not support either a marker or
> an
> > > offset (obviously not both) on the API request?  It appears that
> > > sqlalchemy has offset support.
> > > 
> > > Also, it seems that cinder at least looks for the offset parameter
> > > (but ignores it).  Does this mean that it was supported at one
> time
> > > but later the support was removed?
> > >  https://github.com/openstack/cinder/blob/master/cinder/api/v2/
> > volumes.py#L214
> > > 
> > > Thanks for the information.
> > 
> > Hail to thee, stranger! Thou hast apparently not passed into the
> cave of
> > marker/offset before!
> > 
> > I humbly direct you to buried mailing list treasures which shall
> > enlighten you!
> > 
> > This lengthy thread shall show you how yours truly was defeated in
> > written combat by the great Justin Santa Barbara, who doth exposed
> the
> > perils of the offset:
> > 
> > http://www.gossamer-threads.com/lists/openstack/dev/2803
> > 
> > A most recent incantation of the marker/offset wars is giveth here:
> > 
> >
> http://lists.openstack.org/pipermail/openstack-dev/2013-November/018861.html
> > 
> > Best of days to you,
> > -jay
> 
> Jay:
> 
> Thanks for the feedback and the history on this topic. 

No probs. Thanks for indulging my Renaissance binge this morning.

> I understand that the limit/marker
> approach is superior when simply traversing all of the pages. However,
> consider the
> following:
> 
> - User knows that there are 1000 items (VMs, volumes, images, really
> doesn't matter)
> - User knows that the item that they want is in roughly the middle of
> the data set (assume
> everything is sorted by display name)
> - User cannot remember the exact name so a filter will not help and
> changing the sort 
> direction will not help (since the item they want it is in the middle
> of the dataset)
> - User supplies an offset of 500 to jump into the middle of the data
> set
> - User then uses the marker approach to traverse the pages from this
> point to find the
> item that they want
> 
> In this case the offset approach is not used to traverse pages so
> there are no issues with
> missing an item or seeing a duplicate.

I guess I wonder how common that use case is, actually. I can't remember
ever running into a user who asked for such a thing, but perhaps that's
just me.

> Why couldn't the APIs support either marker or offset on a given
> request? 

I think for two reasons:

1) Would be potentially quite confusing to the user
2) We're lazy? :)

> Also, to encourage
> the use of marker instead of offset, the next/previous links on any
> request with an offset
> supplied should contain the appropriate marker key values -- this
> should help discourage
> simply increasing the offset when traversing the pages.

Well, yes, this already happens today in the projects that support
pagination, AFAIK.

Best,
-jay

> I realize that if only one solution had to be chosen, then
> limit/marker would always win
> this war. But why can't both be supported?
> 
> Thanks,
> 
> Steven Kaufer
> 
> > 
> > 
> > _______________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > 
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev