Open Stack

On Thu, May 26, 2011 at 9:56 AM, Michael Barton
<mike-launchpad at weirdlooking.com> wrote:
> On Wed, May 25, 2011 at 2:40 PM, Jay Pipes <jaypipes at gmail.com> wrote:
>
>> The pagination in Swift is not consistent. Inserts into the Swift
>> databases in between the time of the initial query and the requesting
>> the "next page" can result in rows from the original first page
>> getting on the new second page.
>
> No, you only get records not on the first page, because you're sending
> a marker of the last item from the first page.  Though even if that
> were the case, I wouldn't do very much work to try and provide some
> sort of point-in-time consistent view of the database for pagination.

Pagination after the page you are on is not consistent. Meaning, if I
do a query, go to page 2, I can refresh and the contents of the page
may change. That is not a consistent view of data, and that's what I
was referring to. Sorry if that was confusing...

>> On Wed, May 25, 2011 at 3:32 PM, Greg Holt <gholt at rackspace.com> wrote:
>>> select w from x where y > marker order by y limit z
>
>> LIMIT X OFFSET Y clause. Your query above would return ALL the rows
>> that match WHERE y > marker. That's not what we want. We want a
>> segment of those rows.
>
> He had a limit clause in there.
>
> The reason we usually shy from offsets is they don't scale.  I don't
> know what cardinality you're expecting on these tables, but if you're
> querying for an offset of a million, offset's gotta go count a million
> records before it can return any results.  For a marker query, it can
> just do an index lookup.

Sure, I'll grant you that. Not disagreeing with you on the scalability
issues around OFFSET (at least as implemented in most RDBMS).

My point was about consistent view of data sets and ensuring that page
X of a result set doesn't change over time.

But, it sounds like folks aren't really concerned about the
consistency of the view as much as the scalability concerns, and I'm
perfectly cool with ditching OFFSET in favour of a last-record marker.

-jay