couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Bryan <jbr...@cashnetusa.com>
Subject Re: Write Performance
Date Thu, 08 Jan 2009 00:55:49 GMT
Chris Anderson wrote:
> On Wed, Jan 7, 2009 at 4:37 PM, Josh Bryan <jbryan@cashnetusa.com> wrote:
>   
>> Hi,
>>
>> I am looking into CouchDB as a solution to store a bunch (approx 70
>> million) archived documents.  While planning for the import process, I
>> did some benchmarking to figure out how long the import will take.  I
>> get about 50-70 inserts per second on average.  However, when I looked
>> for the bottleneck, I couldn't figure it out.  I am connected to the
>> database via a fast lan and can verify that the network is not
>> saturated.  I can also verify that disk IO is not saturated.  The only
>> clue is that of the 4 cpus on the server, it seems that only one is
>> getting fully loaded.  Also, of the 5 erlang processes I can see
>> running, only one of them seems to be getting most of the cpu time.  I
>> know that erlang is built with smp enabled, so if it is cpu bound, why
>> can't it make use of the other 3 processors?
>>
>> I thought that perhaps there was some internal write lock issue per
>> database that allowed only one thread to write to a db at a time, so I
>> tried running the benchmarks while hitting multiple databases, but still
>> got the same write rate across the databases.  Is there some globally
>> shared resource in couchdb that limits all writes to a single thread?
>>
>> Thanks,
>> Josh
>>
>>     
>
> Before we can help you diagnose the performance you're seeing, could
> you tell us the version of CouchDB and the version of Erlang that you
> are using? It wouldn't hurt to describe the hardware in more detail
> either.
>
>   
I am seeing similar results on two systems. 

System 1:
Quad core Intel(R) Xeon(R) CPU 5160  @ 3.00GHz
2 GB ram
Linux 2.6.18-4  -- Debian Lenny
Erlang (BEAM) emulator version 5.6.3 [source] [64-bit] [smp:4]
[async-threads:0] [kernel-poll:false]
couchdb - Apache CouchDB 0.8.0-incubating

System 2:
Intel(R) Pentium(R) D CPU 3.00GHz
3 GB ram
Erlang (BEAM) emulator version 5.5.5 [source] [async-threads:0]
[kernel-poll:false]
couchdb - Apache CouchDB 0.9.0a724455-incubating

Thanks



Mime
View raw message