couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Göran Krampe <go...@krampe.se>
Subject Re: Possible bug in indexer... (really)
Date Sun, 05 Jul 2009 00:16:36 GMT
Adam Kocoloski wrote:
> Not sure if it's described, but it is by design.  The reduce function 
> executes when the btree is modified.  We can't afford to cache KVs from 
> an index update in memory regardless of size; we have to set some 
> threshold when we flush them to disk.

And I presume you can't write KVs *without* doing the reduce?

When I wrote "described" I am referring to the blog post by Ricky Ho 
btw. It seems to imply a strict ordering, map -> reduce -> rereduce. IIRC.

> I think the fundamental question is why the flush operations were 
> occurring so frequently the second time around.  Is it because you were 
> building up a largish hash for the reduce value?  Probably.  
> Nevertheless, I'd like to have a better handle on that.

Yeah, well, I am on vacation now - but some other guys are not. We could 
of course start by trying to rewrite this the Right Way first as Chris said.

I am curious if it can be done using grouping because we dismissed 
grouping due to its relatively slow performance (it runs lots of reduces 
at query time IIRC) :)

Btw, the solution used now DOES return the map for a full year in about 
230 ms, including parsing on client side. So query time was perfectly 
fine, but view generation was not. This shows to me that it *can* work.

regards, Göran


Mime
View raw message