incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Lehnardt <...@apache.org>
Subject Re: using mutliple cores for view computation
Date Fri, 20 Mar 2009 10:59:11 GMT

On 20 Mar 2009, at 10:41, Anand Chitipothu wrote:

>> Can you check iostat to see how fast data gets passed to the disk?
>> If we're saturating your write speed already, there's no need to go
>> multi process.
>>
>> (I expect that we don't saturate the disk just yet, so yeah there's  
>> room
>> for improvement, but I don't think disk is fast enough for 4 cores  
>> spitting
>> out data).
>
> Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
> sda1              1.43        33.72        60.85  129287651  233314088
>
> It is surely not saturated by I/O. %iowait is less than 1.0.
>
> The view computation is running from more than 5 hours and I would
> like to see the view server use all the available cores and it will be
> even better it if is possible add more nodes to assist the view
> computation.

If you look into the archives, there are a buch of things that are still
open for speeding this up. We're not there yet.

If you want to have more nodes helping, you need the data on these
cores as well. All speedup might be eaten by data-transfer time.

There are also tricks to make views compute faster, can you show your
m/r function and how large your documents are?

Cheers
Jan
--


Mime
View raw message