couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ross Bates <rba...@gmail.com>
Subject Re: Array being chunked in reduce
Date Thu, 30 Apr 2009 15:26:30 GMT
Thanks Paul - that does help me understand the behavior I'm seeing.

I'm new to couchdb and so far I'm really enjoying it. I've mentally made the
leap from rdb to the doc store and mapping model, but one thing I still
struggle with is discovering use cases for the reduce function. Most
examples are things like sum and count across attributes which could be
handled client side as well. As with anything new though, it's all part of
the learning curve.

I'll keep hacking at it and learning from the list. Thanks again for the
help!





On Wed, Apr 29, 2009 at 2:28 PM, Paul Davis <paul.joseph.davis@gmail.com>wrote:

> Ross Bates wrote:
>
>> Hi All  - I have about 600 of documents with 3 attributes:
>>
>> parent, child, level
>>
>> When I run the following map/reduce everything looks correct at first in
>> that I get an array back where the key is the level and the value is a
>> collection of arrays that have share the same level value. So if there
>> were
>> 5 unique levels there would be 5 rows
>>
>> function(doc) {
>>    emit( parseInt(doc.level), [doc.child,doc.parent]);
>> };
>>
>>
>> function(a,b){
>> return b;
>> }
>>
>>
>> The problem is that when I run this on a larger data set the value array
>> starts getting chunked into arbitrary sizes (50, 83, etc....) When I say
>> chunked, the row has a bunch of additional nested arrays to store the
>> key/value.
>>
>> Is there some sort of max array size imposed by couch db?
>>
>> Many Thanks,
>> Ross
>>
>>
>>
> Ross,
>
> Firstly, the chunked bits you're seeing is couchdb running rereduce on your
> reduce output. The full signature of a reduce function is:
>
>   function(keys, values, rereduce)
>
> Where rereduce is a boolean that flags if the function is being called on
> its own output.
>
> Also, the function as you have it would eventually tip over as its
> returning too much data. For your particular use case I would suggest
> omitting the reduce function entirely and then doing a selection on the map
> and any post-processing client side.
>
> HTH,
> Paul Davis
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message