couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From J Chris Anderson <jch...@gmail.com>
Subject Re: Getting reduce overflow error
Date Thu, 17 Jun 2010 15:10:33 GMT

On Jun 17, 2010, at 7:12 AM, afters wrote:

> I received 'reduce_overflow_error' when querying views, after entering
> largish (2-4kb) docs to couch. The problem was due to a specific view, and
> when I set 'reduce_limit = false' it went away.
> 
> Specifically, each of my documents is related to a logical entity in my app,
> and the purpose of this view is to aggregate information per entity - it
> maps the documents by the entities to which they are related, and reduces
> the results (with group=true) so that each entity ends up with its own
> document.
> 
> The reduce mainly does counts and sums, but it may also accept quite a bit
> of text (~4kb) from one of the documents it reduces. I make sure that these
> blobs of text don't pile up during reduce and rereduce - not more than one
> will be kept at each stage.
> 
> Is 4kb too big to keep when reducing? or maybe the problem is that the
> product of the reduce can be equal to or larger than any of the docs it
> reduces?
> 

The reduce-limit is a general heuristic, because some very bad reduces will actually grow
asymptotically so that the full reduce contains as much data as the entire group=true reduce.
It sounds like yours is OK (large but not growing) so you are probably fine (although keeping
4kb of stuff in the intermediate reduction value storage is going to kill performance. Any
way to break it up and maybe use the reduce to know which document to query to get the big
blob of text?

Chris 

> Thanks,
> a.


Mime
View raw message