incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Anderson <jch...@apache.org>
Subject Re: sort by reduce value
Date Mon, 17 Aug 2009 22:21:11 GMT
On Mon, Aug 17, 2009 at 3:09 PM, Adam Wolff<awolff@gmail.com> wrote:
> I'm not sure I get this. If the results were ordered locally, you could do
> merge sort for the global set. Mergesort has the advantage that its resource
> consumption scales linearly with the size of the result set. Wha'd I miss?

The reductions are ordered by key, if you want to sort them by
reduction value, you have to do it on the global reduce (eg, you can't
just sort by reduce-value on the leaf nodes). It's obvious but groking
it makes a lot of about clustered views (and some Couch feature
choices, like no built-in sort by value) clearer.

> A
>
> On Thu, Aug 13, 2009 at 1:14 PM, Chris Anderson <jchris@apache.org> wrote:
>
>> On Wed, Aug 12, 2009 at 11:40 PM, Brian Candler<B.Candler@pobox.com>
>> wrote:
>> > On Mon, Aug 10, 2009 at 01:56:49PM -0700, Chris Anderson wrote:
>> >> The reason why CouchDB can't do the sorting by value inside itself is
>> >> that on a multi node cluster, an individual couch can't know the
>> >> global ordering. We have a commitment to ensuring that CouchDB's API
>> >> remains invariant regardless of the size of the underlying cluster.
>> >>
>> >> So you basically have to copy the cluster-wide reduce values to an
>> >> intermediate storage mechanism (which itself can be a cluster) before
>> >> you can sort them by value.
>> >
>> > Not sure I understand. If this is a sharded cluster, then each node could
>> > sort its own subset of results; then it's very simple and efficient to
>> merge
>> > the already-sorted results together.
>> >
>>
>> Exactly!
>>
>> Except that in order to get a global ordering for group reduces, you
>> have to do the ordering after the merge, not before.
>>
>>
>>
>> --
>> Chris Anderson
>> http://jchrisa.net
>> http://couch.io
>>
>



-- 
Chris Anderson
http://jchrisa.net
http://couch.io

Mime
View raw message