Open Stack

On 14/08/2013, at 7:40 AM, Jay Pipes <jaypipes at gmail.com> wrote:

> On 08/13/2013 05:04 PM, Gabriel Hurley wrote:
>> I have been one of the earliest, loudest, and most consistent PITA's about pagination, so I probably oughta speak up. I would like to state three facts:
>> 
>> 1. Marker + limit (e.g. forward-only) pagination is horrific for building a user interface.
>> 2. Pagination doesn't scale.
>> 3. OpenStack's APIs have historically had useless filtering capabilities.
>> 
>> In a world where pagination is a "must-have" feature we need to have page number + limit pagination in order to build a reasonable UI. Ironically though, I'm in favor of ditching pagination altogether. It's the lowest-common denominator, used because we as a community haven't buckled down and built meaningful ways for our users to get to the data they really want.
>> 
>> Filtering is great, but it's only 1/3 of the solution. Let me break it down with problems and high level "solutions":
>> 
>> Problem 1: I know what I want and I need to find it.
>> Solution: filtering/search systems.
> 
> This is a good place to start. Glance has excellent filtering/search capabilities -- built in to the API from early on in the Essex timeframe, and only expanded over the last few releases.
> 
> Pagination solutions should build on a solid filtering/search functionality in the API, where there is a consistent sort key and direction (either hard-coded or user-determined, doesn't matter).
> 
> Limit/offset pagination solutions (forward and backwards paging, random skip-to-a-page) are inefficient from a SQL query perspective and should be a last resort, IMO, compared to limit/marker. With some smart session-storage of a page's markers, backwards paging with limit/marker APIs is certainly possible -- just store the previous page's marker.

Not just the previous page's marker, but the marker of every previous page since we would like to be able to click the previous button more than once. Any previous markers we store are also likely to become stale pretty quickly. And all this is based on the assumption that the user's session even started at the first 'page' - it could be they followed a link from elsewhere in Horizon or the greater internet.

I completely agree with Dolph that this is something the client shouldn't need to care about at all. The next/prev links returned with each page of results should hide all of this. next/prev links also make it trivial for the client to discover whether there's even a next page at all, since we don't want to make a user click a link to go to an empty page.

Having said that, I think we can improve the current marker/limit system without hurting performance if we split the marker into 'before' and 'after' parameters. That way all the information needed to go forward or backwards is included in the results for the current page. Supporting 'before' should be as simple as reversing the sort order and then flipping the order of the results.


Kieran


> 
>> Problem 2: I don't know what I want, and it may or may not exist.
>> Solution: tailored discovery mechanisms.
> 
> This should not be a use case that we spend much time on. Frankly, this use case can be summarized as "the window shopper scenario". Providing a quality search/filtering mechanism, including the *API* itself providing REST-ful discovery of the filters and search criteria the API supports, is way more important...
> 
>> Problem 3: I need to know something about *all* the data in my system.
>> Solution: reporting systems.
> 
> Sure, no disagreement there.
> 
>> We've got the better part of none of that.
> 
> I disagree. Some of the APIs have support for a good bit of search/filtering. We just need to bring all the projects up to search speed, Captain.
> 
> Best,
> -jay
> 
> p.s. I very often go to the second and third pages of Google searches. :) But I never skip to the 127th page of results.
> 
> > But I'd like to solve these issues. I have lots of thoughts on all of those, and I think the UX and design communities can offer a lot in terms of the usability of the solutions we come up with. Even more, I think this would be an awesome working group session at the next summit to talk about nothing other than "how can we get rid of pagination?"
>> 
>> As a parting thought, what percentage of the time do you click to the second page of results in Google?
>> 
>> All the best,
>> 
>>     - Gabriel
>> 
>> 
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> 
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev