incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dave Cottlehuber <d...@muse.net.nz>
Subject Re: Reduces out of order?
Date Sat, 18 Jun 2011 02:18:35 GMT
On 18 June 2011 14:07, Sean Copenhaver <sean.copenhaver@gmail.com> wrote:
> Hmmm... I don't think it's a good idea to have an order sensitive reduce function. Also
I think you are thinking about the actual view index which is ordered by the key you emit
from the map function.
>
> I believe couch will reduce each b-tree node worth of the view and store those, then
rereduce those and parts of nodes as needed so your reduced queries only do part of the work.
>
> I hope that made some sense.
>
>
>
> On Jun 17, 2011, at 9:17 PM, Charles Lehner <celehner1@gmail.com> wrote:
>
>> Hi,
>>
>> I have a reduce function that is sensitive to row order. My understanding is that
reduces are supposed to get their rows sorted by key. But this one is getting them out of
order; in fact, the order changes inconsistently between replications.
>>
>> I'm seeing this on both 1.0.2 and trunk. Is this the expected behavior?
>>
>> Thanks,
>> Charles
>

Sean's spot on; MR should not be sensitive to row order; that's kinda
the point of being able to divide and conquer arbitrarily is that the
result is identical and independent of the order of processing.

http://wiki.apache.org/couchdb/Introduction_to_CouchDB_views#Restrictions_on_map_and_reduce_functions

I'm not sure what you meant by between replications, but after more
data is added to the couch, impacted b-tree leaf and intermediate
nodes get re-written. As its append-only, this continues right to the
top of the tree. So your "row order" changes each time more data is
added, potentially.

Perhaps a list might be more appropriate for your post-processing
sorting? or client side?

A+
Dave

Mime
View raw message