incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Wertheim <dan...@wertheim.se>
Subject Re: Regarding "Skip as fast as startkey"
Date Tue, 01 Apr 2014 17:22:17 GMT
Pagination of data sets for me is not only for humans sitting and clicking
next, but also e.g. to batch process a certain range as well. But yes, for
a GUI solution where someone probably will not "go that high" I guess I
agree.

Do I get it right, only doing skip and limit and not combining it with
startkey & startkey_docid is more for taking the "easy way" out?

Thanks,

//Dan


On 1 April 2014 17:36, Alexander Shorin <kxepal@gmail.com> wrote:

> Thanks for additions, Garren
>
> You remind me the important edge case: using just startkey_docid
> pagination isn't always enough. For instance, both Futon and Fauxton
> with older pagination are affected to the issue when the last row in
> fetched view result contains the same docid and key as were used for
> current request.
>
> https://issues.apache.org/jira/browse/COUCHDB-2192
>
> Ironically, it could be only solved by using mixed approach: using
> startkey_docid as primary pagination method for performance and
> fallback to skip one for the COUCHDB-2192 edge case. Hopefully, for
> small numbers of skip there is no any performance degradation could be
> noticed.
>
> --
> ,,,^..^,,,
>
>
> On Tue, Apr 1, 2014 at 7:18 PM, Garren Smith <garren@apache.org> wrote:
> > Alexander raises some good points. I've been working a lot on
> implementing pagination with Couchdb and I find there are tradeoffs for
> using either skip or startkey.
> > The new version of pagination that will land in Fauxton soon (
> https://github.com/apache/couchdb/pull/194) uses skip.
> > This is not ideal in all cases as mentioned in the reasons in this mail.
> However its not that bad for low values for skip. I found for skip values
> less than 100000, the response is not that bad.
> > Depending on your user interface or problem you are trying to solve, how
> many people are paginating from page 1 to a massive page where your skip
> value would be > 100 000? So I think its a fair tradeoff.
> > I know Alexander would disagree.
> >
> > Using startkey and startkey_docid is fine. You have to do a lot more
> checking of the values and make sure you do the correct json encoding. You
> also have to use startkey_docid for views and then startkey for _all_docs.
> > You also get an edge case where you are forced to use skip if the keys
> for the view are all the same.
> >
> > Going forward with Fauxton we will be implementing a new pagination
> algorithm that uses startkey and startkey_docid but then drops down to skip
> for the above mentioned edge case.
> >
> > Cheers
> > Garren
> >
> >
> > On 01 Apr 2014, at 5:04 PM, Alexander Shorin <kxepal@gmail.com> wrote:
> >
> >> There are two issues with limit&skip pagination:
> >>
> >> 1. Performance, as you already noticed with your benchmark. However,
> >> this performance issue is only actual when you made first request with
> >> high skip value. If you'll increase it slowly, request time wouldn't
> >> be too much (but still request time would be too high).
> >>
> >> 2. Design: http://www.reddit.com/comments/1ae0tl
> >>
> >> I have a draft of updates for this article telling why you should stay
> >> with startkey method. Will submit it soon.
> >>
> >>
> >> --
> >> ,,,^..^,,,
> >>
> >>
> >> On Tue, Apr 1, 2014 at 6:12 PM, Daniel Wertheim <daniel@wertheim.se>
> wrote:
> >>> Was looking at this:
> https://issues.apache.org/jira/browse/COUCHDB-1076
> >>>
> >>> Which in the final comment states: "As far as I'm aware, skip is
> >>> equivalently fast to a startkey search"
> >>>
> >>> Glanced at the latest documentation, which has this pre v.1.2 vs after
> >>> v1.2:
> >>>
> http://couchdb.readthedocs.org/en/latest/couchapp/views/pagination.html
> >>>
> >>> Did a quick test with 1.700.543  docs. CouchDb v1.5 Windows. Key is an
> >>> integer from 1 to 1.700.543
> >>>
> >>> 13000ms for limit=10&skip=500000
> >>> 48ms with startkey, startkeydocid, skip, limit
> >>>
> >>> Seems to me that the documents should still say "Don't use this" or
> what am
> >>> I missing?
> >>>
> >>> //Daniel
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message