couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Charles Lehner <celehn...@gmail.com>
Subject Re: Reduces out of order?
Date Sat, 18 Jun 2011 21:45:49 GMT
Thanks for addressing my question, Dave and Sean. It looks like I can't do what I was trying
to do, which is to store deltas as documents and then use a time-ordered view to apply the
deltas and see the current state of the system, or its state at a point in the past.

I could do this with a list function, but that would require iterating through all the documents
for each request. More likely I'll make something custom in node.js.

Charles

On Jun 17, 2011, at 10:18 PM, Dave Cottlehuber <dave@muse.net.nz> wrote:

> On 18 June 2011 14:07, Sean Copenhaver <sean.copenhaver@gmail.com> wrote:
>> Hmmm... I don't think it's a good idea to have an order sensitive reduce function.
Also I think you are thinking about the actual view index which is ordered by the key you
emit from the map function.
>> 
>> I believe couch will reduce each b-tree node worth of the view and store those, then
rereduce those and parts of nodes as needed so your reduced queries only do part of the work.
>> 
>> I hope that made some sense.
>> 
>> 
>> 
>> On Jun 17, 2011, at 9:17 PM, Charles Lehner <celehner1@gmail.com> wrote:
>> 
>>> Hi,
>>> 
>>> I have a reduce function that is sensitive to row order. My understanding is
that reduces are supposed to get their rows sorted by key. But this one is getting them out
of order; in fact, the order changes inconsistently between replications.
>>> 
>>> I'm seeing this on both 1.0.2 and trunk. Is this the expected behavior?
>>> 
>>> Thanks,
>>> Charles
>> 
> 
> Sean's spot on; MR should not be sensitive to row order; that's kinda
> the point of being able to divide and conquer arbitrarily is that the
> result is identical and independent of the order of processing.
> 
> http://wiki.apache.org/couchdb/Introduction_to_CouchDB_views#Restrictions_on_map_and_reduce_functions
> 
> I'm not sure what you meant by between replications, but after more
> data is added to the couch, impacted b-tree leaf and intermediate
> nodes get re-written. As its append-only, this continues right to the
> top of the tree. So your "row order" changes each time more data is
> added, potentially.
> 
> Perhaps a list might be more appropriate for your post-processing
> sorting? or client side?
> 
> A+
> Dave


Mime
View raw message